Training models

Models
4

Direct answer and script:

You cannot train directly on a GGUF file.

Convert GGUF to a supported PyTorch or Hugging Face format, train, then quantize and convert back to GGUF.

Example:

  1. Download or export model in HuggingFace/transformers format, not GGUF.
  2. Train/fine-tune with your chosen framework (e.g. unsloth, transformers, etc).
  3. Convert the resulting checkpoint to GGUF after training.

For Unsloth, see: Llama 4 - Finetune & Run with Unsloth

Solution provided by Triskel Data Deterministic AI.

1 Like