I am trying to instruct fine tune Llama 3.1 8B base. Tried ms-swift and unsloth. Spent hours in training and now sometimes it can stop with the correct eos token (<|eot_id|>). But sometimes it continues generation. Do I have to do more training or am I doing something wrong?
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Fine-tuning don't work / bad results | 3 | 1189 | April 22, 2024 | |
| Setting `pad_token_id` to `eos_token_id`:128001 for open-end generation | 5 | 151 | October 16, 2024 | |
| Question from rookie | 1 | 136 | May 9, 2024 | |
| Help with preparing train data for fine-tuning llama 3.1 instruct model? | 0 | 30 | October 27, 2024 | |
| Bad Performance Finetuning Llama Chat and Instruct Models on GSM8K | 0 | 83 | November 19, 2024 |