I want to merge my PEFT adapter model with the base model and make a fully new model

Beginners
1

As the title said, I want to merge my PEFT LoRA adapter model (ArcturusAI/Crystalline-1.1B-v23.12-tagger) that I trained before with the base model (TinyLlama/TinyLlama-1.1B-Chat-v0.6) and make a fully new model.

And I got this code from ChatGPT:

from transformers import AutoModel, AutoConfig

# Load the pretrained model and LoRA adapter

pretrained_model_name = "TinyLlama/TinyLlama-1.1B-Chat-v0.6"

pretrained_model = AutoModel.from_pretrained(pretrained_model_name)

lora_adapter = AutoModel.from_pretrained("ArcturusAI/Crystalline-1.1B-v23.12-tagger")

# Assuming the models have the same architecture (encoder, decoder, etc.)

# Get the weights of each model

pretrained_weights = pretrained_model.state_dict()

lora_adapter_weights = lora_adapter.state_dict()

# Combine the weights (adjust the weights based on your preference)

combined_weights = {}

for key in pretrained_weights:

combined_weights[key] = 0.8 * pretrained_weights[key] + 0.2 * lora_adapter_weights[key]

# Load the combined weights into the pretrained model

pretrained_model.load_state_dict(combined_weights)

# Save the integrated model

pretrained_model.save_pretrained("ArcturusAI/Crystalline-1.1B-v23.12-tagger-fullmodel")

And I got this error:

---------------------------------------------------------------------------

OSError                                   Traceback (most recent call last)

<ipython-input-1-d2120d727884> in <cell line: 6>()
      4 pretrained_model_name = "TinyLlama/TinyLlama-1.1B-Chat-v0.6"
      5 pretrained_model = AutoModel.from_pretrained(pretrained_model_name)
----> 6 lora_adapter = AutoModel.from_pretrained("ArcturusAI/Crystalline-1.1B-v23.12-tagger")
      7 
      8 # Assuming the models have the same architecture (encoder, decoder, etc.)

1 frames

/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, config, cache_dir, ignore_mismatched_sizes, force_download, local_files_only, token, revision, use_safetensors, *model_args, **kwargs)
   3096                             )
   3097                         else:
-> 3098                             raise EnvironmentError(
   3099                                 f"{pretrained_model_name_or_path} does not appear to have a file named"
   3100                                 f" {_add_variant(WEIGHTS_NAME, variant)}, {TF2_WEIGHTS_NAME}, {TF_WEIGHTS_NAME} or"

OSError: ArcturusAI/Crystalline-1.1B-v23.12-tagger does not appear to have a file named pytorch_model.bin, tf_model.h5, model.ckpt or flax_model.msgpack.

I have no idea what I did wrong there, I would appreciate if anyone can teach me what how to fix it, or am I going for a completely wrong direction? Thank you.

1 Like
2

Have you tried the method merge_and_unload from PeftModel ad shown in this thread Help with merging LoRA weights back into base model :-) - #7 by accOne996795

3

merge_and_unload is not compatible with SafeTensors, only with bin files and OP doesn’t seem to have adapters in bin format.