Correct way to load multiple LoRA adapters for inference

🤗Transformers
3

Thanks for the reply! I tried this and it works perfectly. But, when I try to save the model and load it from local directory, I get the error ValueError: Can't find 'adapter_config.json' at '/path/to/model'. I have tried pushing the model to hub and then loading it, still the same error. I can see there is no adapter_config.json at the path. The json files are actually inside new directories for the adapters.

The file structure is like this:

model
|____adapter_1
|    |_____adapter_config.json
|    |_____adapter_model.safetensors
|____adapter_2
|    |_____adapter_config.json
|    |_____adapter_model.safetensors
|____special_tokens_map.json
|____tokenizer.json
|____tokenizer.config.json
|____vocab.txt
|____README.md

I am trying to load the model with adapters like this (the code is from this discussion):

outputs = "/path/to/model"
adapter_1 = "/path/to/model/adapter_1"
adapter_2 = "/path/to/model/adapter_2"

adapter_1_config = PeftConfig.from_pretrained(adapter_1)
adapter_2_config = PeftConfig.from_pretrained(adapter_2)

base_model = AutoModelForSequenceClassification.from_pretrained(model_name, num_labels=2, output_hidden_states=False)

peft_model = PeftModelForSequenceClassification.from_pretrained(base_model, outputs, num_labels=2)
peft_model.load_adapter(adapter_1)
peft_model.load_adapter(adapter_2)
1 Like