As best as I can tell, the LoraModel
merge_and_unload attribute (peft/lora.py at main · huggingface/peft · GitHub) merges LoRA weights back into the main model.
However, I am having trouble getting a LoraModel type from my PeftModelForCausalLM. My current workflow is to define a pretrained model, define a LoraConfig, and use the
get_peft_model function to being training. This works great, but I want to be able to merge the weights back into the base model and save.
My working assumption is that I need to either convert my PeftModelForCausalLM into a LoraModel or initialize the model as a LoraModel prior to training. However, when I copy the example in the LoraModel docstring (peft/lora.py at main · huggingface/peft · GitHub), I get an
TypeError (TypeError: LoraModel.init() missing 1 required positional argument: ‘adapter_name’). When I try passing a “lora” as a adapter name, I get another error.
I think that I am fundamentally thinking about this in the wrong way and would love some pointers. Both Google and Copilot chat have not been able to solve my problem.
I figured this out. The solution is quite simple.
A PeftModelForCausalLM actually inherits the LoraModel methods, so you can call
merged_model = merged.merge_and_unload() to get back a base model with the LoRA weights applied.
My IDE would not autocomplete
merge_and_upload, so I assumed the method wasn’t available. I still don’t need in the code where this method is inherited and would love for someone to point this out to me if feeling charitable.
Looking to get a model file from a base + lora myself. Can you explain in more detail how you were able to do it?
Try this. Basic steps are to:
1/ load the base model
2/ train the base model
3/ save the LoRA adapter
4/ reload the base model at half/full precision
5/ merge the LoRA weights with the base model
base_model = AutoModelForCausalLM.from_pretrained(“base_model”, load_in_8bit=True, torch_dtype=torch.float16, device_map=“auto”)
base_model = prepare_model_for_int8_training(base_model)
peft_model = get_peft_model(base_model, peft_config)
training_args = TrainingArguments()
trainer = Trainer()
peft_model.save_pretrained(lora_adapter, save_adapter=True, save_config=True)
model_to_merge = PeftModel.from_pretrained(AutoModelForCausalLM.from_pretrained(base_model).to(“cuda”), lora_adapter)
merged_model = model_to_merge.merge_and_unload()
These are the correct steps to create a model version form a base model and my train?
A related question, why the model size in disk is almost doubling after merging even when the number of parameters remaining the same?