However, I am having trouble getting a LoraModel type from my PeftModelForCausalLM. My current workflow is to define a pretrained model, define a LoraConfig, and use the get_peft_model function to being training. This works great, but I want to be able to merge the weights back into the base model and save.
My working assumption is that I need to either convert my PeftModelForCausalLM into a LoraModel or initialize the model as a LoraModel prior to training. However, when I copy the example in the LoraModel docstring (peft/lora.py at main · huggingface/peft · GitHub), I get an TypeError (TypeError: LoraModel.init() missing 1 required positional argument: ‘adapter_name’). When I try passing a “lora” as a adapter name, I get another error.
I think that I am fundamentally thinking about this in the wrong way and would love some pointers. Both Google and Copilot chat have not been able to solve my problem.
A PeftModelForCausalLM actually inherits the LoraModel methods, so you can call merged_model = merged.merge_and_unload() to get back a base model with the LoRA weights applied.
My IDE would not autocomplete merge_and_upload, so I assumed the method wasn’t available. I still don’t need in the code where this method is inherited and would love for someone to point this out to me if feeling charitable.
Try this. Basic steps are to:
1/ load the base model
2/ train the base model
3/ save the LoRA adapter
4/ reload the base model at half/full precision
5/ merge the LoRA weights with the base model
6/ save
But when I try to save the model weights on model_to_merge then I still get only the adapter safetensors and not the safetensors for the entire model.
How do i get that?
I see 2 potential reasons for that, unsure if any of them is applicable to your use case as I cannot see your code.
The merged model (base model + LoRA adapters) has the numer of parameters of the base model + the number of parameters of the inserted LoRA adapters. For this reason,# params merged_model > # params base_model, therefore the increase in size
When reloading the model, make sure to provide the same data type as in training to ensure the same size is mantained. For example, if the trained model was loaded in half precision, also the model_to_merge should be loaded in half precision to enforce comparable size e.g:
In the example provided by @dmckinno, the model_to_train is loaded in 8 bits (load_in_8bit=True), and then model_to_merge is loaded in full precision, since no parameter is provided and the default will be used (float32)