I have multiple loras that I’ve trained on a base model and I’ve been attempting to successively merge them into the base model, however there appears to be some issue with the resulting model. Specifically, I believe the issue is with the tokenizer, but I’m uncertain how to fix this issue, as my understanding is that the tokenizer should simply be the same as the base model.
When I load the base model into text-generation-webui and apply all three loras, I get good generations with the expected behavior, however, after merging them together with the below code I get gibberish that seems very clearly to be applying incorrect tokens.
model = AutoModelForCausalLM.from_pretrained(
"unsloth/Mistral-Small-Instruct-2409-bnb-4bit"
)
tokenizer = AutoTokenizer.from_pretrained(
"unsloth/Mistral-Small-Instruct-2409-bnb-4bit"
)
model = PeftModel.from_pretrained(model, "hobie-lora4")
model = model.merge_and_unload()
model.save_pretrained("HobieLLMPart4")
tokenizer.save_pretrained("HobieLLMPart4")
model = PeftModel.from_pretrained(model, "hobie-lora5")
model = model.merge_and_unload()
model.save_pretrained("HobieLLMPart5")
tokenizer.save_pretrained("HobieLLMPart5")
model = PeftModel.from_pretrained(model, "hobie-lora6")
model = model.merge_and_unload()
model.save_pretrained("HobieLLMPart6")
tokenizer.save_pretrained("HobieLLMPart6")