Hi, I am fine-tuning LLAMA on a multiple-choice question-answering (MCQA) dataset. During the training phase, would it be a good approach to trim the model’s output head to just four tokens corresponding to the answer options, so that during the generation phase, the model is constrained to generate only the labels? Are there any alternative strategies I could consider for achieving this?
Related topics
Topic | Replies | Views | Activity | |
---|---|---|---|---|
Finetuning llama for classification | 2 | 826 | January 21, 2025 | |
Llama-2-7b-chat fine-tuning | 4 | 6233 | April 26, 2024 | |
MCQA Model Underfitting the Training Data | 0 | 114 | March 22, 2024 | |
Question-Answering/Text-generation/Summarizing: Fine-tune on multiple answers | 8 | 5132 | November 20, 2021 | |
How does the model generate the answer 1 time? | 0 | 81 | May 19, 2024 |