I am finetuning a model using peft adapters and want to check if a checkpoint that I have saved is good enough. However, I have added an extra token to the vocabulary before fine-tuning, which results in different embedding size. I try to load the model like this: Load base model model = AutoMode…

Loading Peft model from checkpoint leading into size missmatch

nielsr February 5, 2024, 8:40pm 2

Hi,

In case you have added tokens, it’s recommended to use the PeftModel class rather than AutoModelForCausalLM. The former takes into account resizing the embedding matrix.

cc @ybelkada

1 Like

Topic		Replies	Views
Size mismatch error in PEFT fine tuned model 🤗Transformers	4	1331	July 2, 2024
Having trouble loading a fine-tuned PEFT model (CodeLlama-13b-Instruct-hf base) 🤗Transformers	2	4142	October 6, 2024
Size Mismatch error for LLM checkpoint of PEFT model with a resized token embeddings Models	4	269	January 10, 2025
Resize embeddings on Peft model Intermediate	3	333	January 22, 2025
Peft model from pretrained load in 8/4 bit 🤗Transformers	6	17255	October 12, 2023

Loading Peft model from checkpoint leading into size missmatch

Related topics