Performance problems with finetuned model (Llama 2 7B based)

Beginners
1

Hello,

I am trying to finetune Llama 2 7B to enhance its function calling abilities. I am not sure what I am doing wrong, but the model is not really converging while training and the performance, hence, isn’t good at all.

I am using this Jupyter Notebook and tried it with and without quantization.

These are the training and eval/test datasets. I made them, and they are designed to learn the ‘get_current_weather’ function.

As my datasets are small, I picked higher values for r and lora_alpha

r=256, lora_alpha=512

Does anyone have any ideas on how to improve the model? Am I doing something wrong?

I am very new to the topic of finetuning LLMs. So, any feedback or help would be very much appreciated.

2

Hi,

i’d recommend taking a look at the following resources to fine-tune open-source LLMs:

I’d recommend starting with a super tiny dataset and see if the model is able to overfit it.

3

Thanks, I will have a look :slight_smile:

I tried lower values for both before going up. The higher values seemed to work a little better but still not good.

I used a different approach, adding the SFTTrainer to the fine tuning process, which makes things work well.

4

Yes see also my fine-tuning notebook regarding Mistral: Transformers-Tutorials/Mistral/Supervised_fine_tuning_(SFT)_of_an_LLM_using_Hugging_Face_tooling.ipynb at master · NielsRogge/Transformers-Tutorials · GitHub