Help with merging LoRA weights back into base model :-)

Beginners
1

As best as I can tell, the LoraModel merge_and_unload attribute (peft/lora.py at main · huggingface/peft · GitHub) merges LoRA weights back into the main model.

However, I am having trouble getting a LoraModel type from my PeftModelForCausalLM. My current workflow is to define a pretrained model, define a LoraConfig, and use the get_peft_model function to being training. This works great, but I want to be able to merge the weights back into the base model and save.

My working assumption is that I need to either convert my PeftModelForCausalLM into a LoraModel or initialize the model as a LoraModel prior to training. However, when I copy the example in the LoraModel docstring (peft/lora.py at main · huggingface/peft · GitHub), I get an TypeError (TypeError: LoraModel.init() missing 1 required positional argument: ‘adapter_name’). When I try passing a “lora” as a adapter name, I get another error.

I think that I am fundamentally thinking about this in the wrong way and would love some pointers. Both Google and Copilot chat have not been able to solve my problem.

3 Likes
2

I figured this out. The solution is quite simple.

A PeftModelForCausalLM actually inherits the LoraModel methods, so you can call merged_model = merged.merge_and_unload() to get back a base model with the LoRA weights applied.

My IDE would not autocomplete merge_and_upload, so I assumed the method wasn’t available. I still don’t need in the code where this method is inherited and would love for someone to point this out to me if feeling charitable.

3 Likes
3

Looking to get a model file from a base + lora myself. Can you explain in more detail how you were able to do it?

4

Try this. Basic steps are to:
1/ load the base model
2/ train the base model
3/ save the LoRA adapter
4/ reload the base model at half/full precision
5/ merge the LoRA weights with the base model
6/ save

base_model = AutoModelForCausalLM.from_pretrained(“base_model”, load_in_8bit=True, torch_dtype=torch.float16, device_map=“auto”)

base_model = prepare_model_for_int8_training(base_model)

peft_model = get_peft_model(base_model, peft_config)

training_args = TrainingArguments()
trainer = Trainer()
trainer.train()

peft_model.save_pretrained(lora_adapter, save_adapter=True, save_config=True)

model_to_merge = PeftModel.from_pretrained(AutoModelForCausalLM.from_pretrained(base_model).to(“cuda”), lora_adapter)

merged_model = model_to_merge.merge_and_unload()
merged_model.save_pretrained(merged_model)

19 Likes
5

These are the correct steps to create a model version form a base model and my train?

6

A related question, why the model size in disk is almost doubling after merging even when the number of parameters remaining the same?

2 Likes
7

But when I try to save the model weights on model_to_merge then I still get only the adapter safetensors and not the safetensors for the entire model.
How do i get that?

8

I have the same question, why the size difference in the base model and merged model?

1 Like