I am fine tuning fastchat model using lora-peft (8 bit quantization). My dataset is a question answer based dataset. However, after fine tuning the response are not accurate for the maximum time. My dataset is over 500 entries. I tried changing hyper-parameters, but results are same. Is there anyway to improve it? Or my model choice is wrong? As my goal is to generate a chatbot for a custom dataset.
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| A fine tuned Llama2-chat model can't answer questions from the dataset | 0 | 309 | December 20, 2023 | |
| A fine tuned Llama2-chat model can’t answer questions from the dataset | 0 | 246 | December 23, 2023 | |
| Fine tuning GPT2 on persona chat dataset outputs gibberish | 1 | 2737 | April 14, 2021 | |
| Getting wrong response after fine tuning google/flan-t5-small model? | 0 | 484 | April 27, 2023 | |
| Getting wrong response after fine tuning Google/t5-v1_1-base | 0 | 170 | April 17, 2023 |