can we fine tune ggml model or we will have to fine tune original model then convert it to ggml format as mentioned on some github repo,just a beginner doubt?
i finetuned a sharded model but not able to fine tune ggml model getting error TheBloke/Llama-2-7B-GGML does not appear to have a file named pytorch_model.bin, tf_model.h5, model.ckpt or flax_model.msgpack.
is ggml not for fine tuning??
Sadly, it’s not possible to fine tune ggml models yet I believe, only train them from scratch.
Ggml models are basically for inference but it is kinda possible to train your own model from scratch
More things with fine tuning might come later but not now.
I have a quantized Llama 7B GGML model.
I want to experiment by continue pretraining on my data and want to check the before and after perplexity.
Can I train the GGML model?
If yes, then how can I load the model as huggingface transformer model and then train it?
Or is there any other way to load and train the model?
Else there is no way to train GGML models and It is only for the inference?
Yes ggml model is only for inference. There is a way to train it from scratch but that’s probably not what you want to do. Ggml models were supposed to be for llama cpp but not ggml models are kinda useless llama cpp doesn’t support them anymore. They only support gguf models