Peft Model For SequenceClassification failing _is_peft_model

Beginners
1

I am attempting to finetune MixtralForSequenceClassification, and believe I have a peft model. However, when I call trainer.train() I run into the error “ValueError: You cannot perform fine-tuning on purely quantized models. Please attach trainable adapters on top of the quantized model to correctly perform fine-tuning. Please see: Load adapters with 🤗 PEFT for more details”

Here’s what I did to get here, following the steps in a few popular colab example notebooks:

  1. Loaded mixtral using 4bit quantization with a standard config
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16
)
  1. Prepared for kbit training
from peft import prepare_model_for_kbit_training

model.gradient_checkpointing_enable()

model = prepare_model_for_kbit_training(model)
  1. Used a standard LoraConfig with task_type SEQ_CLS
from peft import LoraConfig, get_peft_model

config = LoraConfig(
    r=32,
    lora_alpha=64,
    target_modules=[
        "q_proj",
        "k_proj",
        "v_proj",
        "o_proj",
        "gate_proj",
        "up_proj",
        "down_proj",
        "lm_head",
    ],
    bias="none",
    lora_dropout=0.05,  # Conventional
    task_type="SEQ_CLS",
)

model = get_peft_model(model, config)
print_trainable_parameters(model)

When I check if the model is an instance of a PeftModel, it returns true:

isinstance(model, peft.PeftModel)

And printing the model yields this:

image
image860×162 11 KB

However _is_peft_model is returning false

I followed similar steps for finetuning Llama ~4 months ago and did not run into this error. Doing the exact same thing as some of these colab notebooks I have seen, so not sure what the error is. Perhaps something specific to sequence classification?

Any help would be much appreciated!