Poor results (val_loss) on fine-tuning the NLLB-200-600M with LoRA for French-Wolof translation

[John6666]

Could this possibly be related to the following issue? If so, it is a bug in the PEFT library and seems to be fixed in the latest version.

LLaMa3.1 8B Instruct Prompt Tuning for Text Classification doesn't improve test accuracy

Hello everyone! I am trying to fine-tune LLaMa3.1 8B Instruct for text classification with prompt tuning from peft library (prepending a trainable matrix before input embeddings). For instance, I am using the QNLI dataset to classify whether a question and sentence are in entailment. I have preprocessed the dataset so that each data looks like the following: <|begin_of_text|><|start_header_id|>system<|end_header_id|> Cutting Knowledge Date: December 2023 Today Date: 26 Jul 2024 <|eot_id|><|…