omar8
1
I’m planning to create a fine-tuned version of LLaMA 3 and need some advice on the best approach. I have a few questions:
- Dataset Size: How large does the dataset need to be to notice a meaningful difference in performance?
- Dataset Format: Is the format of the dataset important? Is a CSV file fine?
- Fine-Tuning Methods: What’s the best way to fine-tune LLaMA 3? Has anyone used the Unsloth framework, and would you recommend it?
Hello, Biren here
Answers:
- I would recommend having around 10k+ entries to get a good impact of learning
- I personally recommend JSON format
[
{
"instruction": "Question",
"response": "Answer"
},
]
- I would recommend training it using auto train on a large AWS instance for better accuracy.
OR
Below is the link for unsloth you can used there refrences to train it on colab or desktop
GitHub - unslothai/unsloth: Finetune Llama 3, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory
Hoping this helps…