I am fine tuning fastchat model using lora-peft (8 bit quantization). My dataset is a question answer based dataset. However, after fine tuning the response are not accurate for the maximum time. My dataset is over 500 entries. I tried changing hyper-parameters, but results are same. Is there anyway to improve it? Or my model choice is wrong? As my goal is to generate a chatbot for a custom dataset.
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Fine Tune with/without LORA | 1 | 75 | October 7, 2024 | |
| Bad Performance Finetuning Llama Chat and Instruct Models on GSM8K | 5 | 186 | December 5, 2024 | |
| Facing issues in fine-tuning Vicuna-7b model | 0 | 480 | January 18, 2024 | |
| Repeatedly decoding tokens multiple times after PEFT fine-tuning whisper | 2 | 599 | September 20, 2023 | |
| Poor results (val_loss) on fine-tuning the NLLB-200-600M with LoRA for French-Wolof translation | 3 | 104 | October 1, 2024 |