I wonder how to merge my PEFT adapter with the base model and finally get a new whole model?

John6666 · February 5, 2025, 1:29pm

So the method should be correct, but it seems that there are cases where an error occurs during merging if the model is quantized.

github.com/huggingface/peft

Merge weights Opt model lora adapter

opened 04:39PM - 13 Apr 23 UTC

closed 04:55PM - 13 Apr 23 UTC

arthurangelici

Im trying to convert to a standard transformer Code: ``` import torch fro…m peft import PeftModel import transformers import os, time import tempfile from transformers import AutoModelForCausalLM, AutoTokenizer from peft import PeftModel, PeftConfig from transformers import OPTForCausalLM, AutoTokenizer BASE_MODEL = "facebook/opt-6.7b" LORA_WEIGHTS = "arthurangelici/opt-6.7b-lora-caramelo" peft_model_id = "arthurangelici/opt-6.7b-lora-caramelo" config = PeftConfig.from_pretrained(peft_model_id) model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path, return_dict=True, load_in_8bit=True, device_map='auto') tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path) model = PeftModel.from_pretrained(model, peft_model_id) model = model.merge_and_unload() model.save_pretrained("arthurangelici/opt-6.7b-lora-merged-caramelo") ``` Error: 'PeftModelForCausalLM' object has no attribute 'merge_and_unload' 'LoraModel' object has no attribute 'merge_and_unload' 'OPTForCausalLM' object has no attribute 'merge_and_unload'

Topic		Replies	Views
I want to merge my PEFT adapter model with the base model and make a fully new model Beginners	4	4520	February 5, 2025
How do I merge a lora adapter back into the model weights? Beginners	1	3123	August 23, 2023
Help with merging LoRA weights back into base model :-) Beginners	11	61221	February 6, 2025
SFTTrainer Merge LoRA weights back into base model? Models	0	1595	December 24, 2023
How to unload an adapter in PEFT? 🤗Accelerate	2	3048	January 15, 2024

I wonder how to merge my PEFT adapter with the base model and finally get a new whole model?

Related topics