LoRA Adapter Loading Issue with Llama 3.1 8B - Missing Keys Warning

Beginners
1

Hi everyone,

I’m having trouble loading my LoRA adapters for inference after fine-tuning Llama 3.1 8B. When I try to load the adapter files in a new session, I get a warning about missing adapter keys:

/usr/local/lib/python3.11/dist-packages/peft/peft_model.py:599: UserWarning: Found missing adapter keys while loading the checkpoint: ['base_model.model.model.layers.0.self_attn.q_proj.lora_A.default.weight', 'base_model.model.model.layers.0.self_attn.q_proj.lora_B.default.weight', 'base_model.model.model.layers.0.self_attn.k_proj.lora_A.default.weight',...

The warning continues with many similar missing keys. The model behaves as if it wasn’t fine-tuned at all.

My setup:

  • transformers 4.43.1
  • peft 0.14.0
  • bitsandbytes 0.45.4
  • torch 2.6.0+cu124
  • accelerate 0.31.0

After training, I saved the model and got these files:

  1. adapter_model.safetensors
  2. adapter_config.json
  3. training_args.bin

Here’s how I loaded the model and adapters:

# Load base model
base_model_id = "meta-llama/Llama-3.1-8B"
lora_weights = "path to weights"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16
)

base_model = AutoModelForCausalLM.from_pretrained(
    pretrained_model_name_or_path=base_model_id,
    quantization_config=bnb_config,
    trust_remote_code=True,
    token=True
)

# Set up tokenizer
eval_tokenizer = AutoTokenizer.from_pretrained(
    base_model_id, 
    add_bos_token=True, 
    trust_remote_code=True, 
    use_fast=True
)
eval_tokenizer.pad_token = eval_tokenizer.eos_token

# Load LoRA adapters
ft_model = PeftModel.from_pretrained(base_model, lora_weights, output_loading_info=True)

Any ideas on what might be causing this issue or how to fix it?

Thanks in advance!

2 Likes
2

I think this is probably a warning related to this specification.