Combine base model with my Peft adapters to generate new model

Beginners
1

Hi All,

I am trying to merge my fine-tuned adapters to the base model. With this

torch.cuda.empty_cache()
del model
pre_trained_model_checkpoint = "databricks/dolly-v2-3b"
trained_model_chekpoint_output_folder = "/content/gdrive/MyDrive/AI/Adapters/myAdapter-dolly-v2-3b/"

base_model = AutoModelForCausalLM.from_pretrained(pre_trained_model_checkpoint,
                                  trust_remote_code=True,
                                  device_map="auto"
                                  )
model_to_merge = PeftModel.from_pretrained(base_model,trained_model_chekpoint_output_folder)
del base_model
torch.cuda.empty_cache()

merged_model = model_to_merge.merge_and_unload()

tokenizer = AutoTokenizer.from_pretrained(trained_model_chekpoint_output_folder)

Then

merged_model.save_pretrained('path')

The generated model size is the aprox the double. (5.6Gb to 11Gb) My fine tunning basically add info about 200 examples dataset in Alpaca format.

what am I doing wrong?

2

Is this problem solved? I am having the same issue with Falcon 1b. The number of parameters remaining the same but the model size on disk is doubling from 2.4g to about 4.5g.