I have modified https://huggingface.co/blog/4bit-transformers-bitsandbytes to work with llama-2. It works, but I have some questions. When the training data is created, the function data.map() is called, and it defaults to a batch_size of 1000. The code goes like this: data = load_dataset("Abirat…

Finetuning quantised llama-2 with LoRA

magnus42 September 23, 2023, 9:54am 2

Did the finetuned model work when you ran it? Did it actually improve the performance for this dataset compared to the base model? How many steps did you train it for?

Topic		Replies	Views
ValueError: Expected input batch_size (1024) to match target batch_size (4) Beginners	1	79	January 7, 2025
What is the loss Function when fine-tuning LlamaV2 Models	0	2145	September 19, 2023
LLM training loss fluctuation 🤗Transformers	0	951	November 30, 2023
How to train a LlamaTokenizer? 🤗Tokenizers	22	4032	August 20, 2024
Memory requierements Models	2	398	February 18, 2025

Finetuning quantised llama-2 with LoRA

Related topics