Qlora adapter with smolagents

Beginners
1

Hello - I want to use my qlora adapter with smolagents. Is this possible?

1 Like
2

I think it might work using the method below. If it doesn’t work, you’ll probably need to merge it yourself and save it locally or upload it to HF…
Or, I think it will be addressed if you raise an issue on github. It’s a necessary function.:sweat_smile:

from smolagents import TransformersModel

# model_id = "HuggingFaceTB/SmolLM-135M-Instruct"
model_id = "bunnycore/Llama-3.2-3B-R1-lora" # An adapter with appropriate "base_model:" setting
model = TransformersModel(model_id=model_id)

print(model([{"role": "user", "content": [{"type": "text", "text": "Ok!"}]}], stop_sequences=["great"]))

Tips