I try to learn the basics of smolagents and I got the following big problem - please help!
I am getting the message that I have run out of the free tier for HfApiModel, and I need to buy the paid tier.
How can I use the local model to run with my CodeAgent in smolagents?
1 Like
There are several methods, but you can use TransformersModel instead of HfApiModel and it will work. When using large models, powerful GPUs are required, so please be careful. Well, I think SmolLM below will work with about 1GB of VRAM…
Also, Ollama is faster and uses less VRAM, but I think it will be a little difficult to set up (compared to TransformersModel…).
from smolagents import TransformersModel
model = TransformersModel(model_id="HuggingFaceTB/SmolLM-135M-Instruct")
I tried to run the following code:
from smolagents import TransformersModel
from smolagents import CodeAgent, DuckDuckGoSearchTool, HfApiModel
model = TransformersModel(model_id="HuggingFaceTB/SmolLM-135M-Instruct")
agent = CodeAgent(tools=[], model=model, additional_authorized_imports=['datetime'])
agent.run(
"""
Alfred needs to prepare for the party. Here are the tasks:
1. Prepare the drinks - 30 minutes
2. Decorate the mansion - 60 minutes
3. Set up the menu - 45 minutes
4. Prepare the music and playlist - 45 minutes
If we start right now, at what time will the party be ready?
"""
)
-
I use my Mac M3’s CPU. And it does not produce responses. There is sertanly a bug here in code. Maybe the model is too big, and yes, I don’t have any GPU here at my place. Is the code fine?
-
Another question, let’s say I will buy the PRO subscription. Can I use HuggingFace API also for LLamaIndex and LangGraph? I think, I saw that the API will work with LLamaIndex, so the bigger question - is the API suitable with LangGraph?
Thank you in advance
1 Like
from smolagents import CodeAgent, TransformersModel
model = TransformersModel(model_id="HuggingFaceTB/SmolLM-135M-Instruct")
agent = CodeAgent(tools=[], model=model, additional_authorized_imports=['datetime'], add_base_tools=True)
agent.run(
"""
Alfred needs to prepare for the party. Here are the tasks:
1. Prepare the drinks - 30 minutes
2. Decorate the mansion - 60 minutes
3. Set up the menu - 45 minutes
4. Prepare the music and playlist - 45 minutes
If we start right now, at what time will the party be ready?
"""
)
I think the code is correct, but it seems that the name of HfApiModel has changed (mainly the name) in the latest version of smolagents, so I made some minor corrections there.
Also, it’s a bit slow on the CPU… but it should still work. It’s just incredibly slow.
On a Mac, using MLX or Ollama might speed things up a bit…
Uff ok, CodeAgent is not good with “HuggingFaceTB/SmolLM-135M-Instruct” model. As far as I understand, the reason is that the context window is too small for the default huge prompt of the CodeAgent. So I tried to use another kind of agent - ToolCallingAgent:
from smolagents import DuckDuckGoSearchTool, ToolCallingAgent
from smolagents import MLXModel
model = MLXModel(model_id="HuggingFaceTB/SmolLM-135M-Instruct")
agent = ToolCallingAgent(tools=[DuckDuckGoSearchTool()], model=model)
agent.run("Search for the best music recommendations for a party at the Wayne's mansion.")
agent.run(
"""
what color is the sky?
"""
)
Here, I’ve also inserted the model within the MLXModel class, which works well on my Mac, at least.
And this is the output:
There is a need to adapt, but I am not sure what exactly. The prompt does not help the model? idk. Does the code run in your environment by any chance?..
1 Like
My local environment is Python 3.9, so smolagents doesn’t work locally…
Anyway, I forgot about that bug, or rather, that feature…
from smolagents import DuckDuckGoSearchTool, ToolCallingAgent
from smolagents import MLXModel
model = MLXModel(model_id="HuggingFaceTB/SmolLM-135M-Instruct", max_new_tokens=4096)
agent = ToolCallingAgent(tools=[DuckDuckGoSearchTool()], model=model)
agent.run("Search for the best music recommendations for a party at the Wayne's mansion.")
agent.run(
"""
what color is the sky?
"""
)
Good code example for using Ollama with smolagents.
1 Like
I just posted another solution in the course category, check it out here. This way you’re not tied only to local inference if you don’t have a beefy machine.
1 Like