I try to learn the basics of smolagents
and I got the following big problem - please help!
I am getting the message that I have run out of the free tier for HfApiModel
, and I need to buy the paid tier.
How can I use the local model to run with my CodeAgent
in smolagents
?
There are several methods, but you can use TransformersModel
instead of HfApiModel
and it will work. When using large models, powerful GPUs are required, so please be careful. Well, I think SmolLM
below will work with about 1GB of VRAM…
Also, Ollama
is faster and uses less VRAM, but I think it will be a little difficult to set up (compared to TransformersModel
…).
from smolagents import TransformersModel
model = TransformersModel(model_id="HuggingFaceTB/SmolLM-135M-Instruct")
I tried to run the following code:
from smolagents import TransformersModel
from smolagents import CodeAgent, DuckDuckGoSearchTool, HfApiModel
model = TransformersModel(model_id="HuggingFaceTB/SmolLM-135M-Instruct")
agent = CodeAgent(tools=[], model=model, additional_authorized_imports=['datetime'])
agent.run(
"""
Alfred needs to prepare for the party. Here are the tasks:
1. Prepare the drinks - 30 minutes
2. Decorate the mansion - 60 minutes
3. Set up the menu - 45 minutes
4. Prepare the music and playlist - 45 minutes
If we start right now, at what time will the party be ready?
"""
)
-
I use my Mac M3’s CPU. And it does not produce responses. There is sertanly a bug here in code. Maybe the model is too big, and yes, I don’t have any GPU here at my place. Is the code fine?
-
Another question, let’s say I will buy the PRO subscription. Can I use HuggingFace API also for LLamaIndex and LangGraph? I think, I saw that the API will work with LLamaIndex, so the bigger question - is the API suitable with LangGraph?
Thank you in advance
from smolagents import CodeAgent, TransformersModel
model = TransformersModel(model_id="HuggingFaceTB/SmolLM-135M-Instruct")
agent = CodeAgent(tools=[], model=model, additional_authorized_imports=['datetime'], add_base_tools=True)
agent.run(
"""
Alfred needs to prepare for the party. Here are the tasks:
1. Prepare the drinks - 30 minutes
2. Decorate the mansion - 60 minutes
3. Set up the menu - 45 minutes
4. Prepare the music and playlist - 45 minutes
If we start right now, at what time will the party be ready?
"""
)
I think the code is correct, but it seems that the name of HfApiModel has changed (mainly the name) in the latest version of smolagents, so I made some minor corrections there.
Also, it’s a bit slow on the CPU… but it should still work. It’s just incredibly slow.
On a Mac, using MLX or Ollama might speed things up a bit…
Uff ok, CodeAgent is not good with “HuggingFaceTB/SmolLM-135M-Instruct” model. As far as I understand, the reason is that the context window is too small for the default huge prompt of the CodeAgent. So I tried to use another kind of agent - ToolCallingAgent
:
from smolagents import DuckDuckGoSearchTool, ToolCallingAgent
from smolagents import MLXModel
model = MLXModel(model_id="HuggingFaceTB/SmolLM-135M-Instruct")
agent = ToolCallingAgent(tools=[DuckDuckGoSearchTool()], model=model)
agent.run("Search for the best music recommendations for a party at the Wayne's mansion.")
agent.run(
"""
what color is the sky?
"""
)
Here, I’ve also inserted the model within the MLXModel
class, which works well on my Mac, at least.
And this is the output:
There is a need to adapt, but I am not sure what exactly. The prompt does not help the model? idk. Does the code run in your environment by any chance?..
My local environment is Python 3.9, so smolagents doesn’t work locally…
Anyway, I forgot about that bug, or rather, that feature…
from smolagents import DuckDuckGoSearchTool, ToolCallingAgent
from smolagents import MLXModel
model = MLXModel(model_id="HuggingFaceTB/SmolLM-135M-Instruct", max_new_tokens=4096)
agent = ToolCallingAgent(tools=[DuckDuckGoSearchTool()], model=model)
agent.run("Search for the best music recommendations for a party at the Wayne's mansion.")
agent.run(
"""
what color is the sky?
"""
)