CodeAgent Search Results Don't Match Lesson Output in SmolAgents AI Agents Course

Beginners
3

Update on my issue with CodeAgent / search results:

Before I even posted this thread, I had already spent 10 to 15 hours trying to get the lesson working as intended. Since then, I’ve kept experimenting, but I’m still running into a lot of inconsistent and unreliable behavior that makes it hard to complete the example successfully.

Here’s what I’ve tried:

  • Reworded the prompt many times (from vague to very specific), especially to guide the agent away from tool misuse or hallucinations
  • Tried using other models, both local and hosted
  • Created custom tools to fetch and extract content from web pages
  • Watched the output at each step to debug parsing issues, formatting errors, and tool logic
  • Upgraded my Hugging Face plan after running out of tokens, just to keep testing

Also, just to clarify based on an earlier suggestion: I do have duckduckgo_search installed, so the agent is using the intended search backend and not falling back to mock behavior.

The most frustrating part is that I actually got this working once — the very first day I tried the lesson. But after I ran out of free tokens (possibly during the tools section), I couldn’t reproduce the success. Even after upgrading to a paid plan, the same prompt started failing in multiple ways. That’s when I started trying different models, tweaking prompts, and building fallback tools. Since then:

  • The agent frequently tries to call tools that are either forbidden or hallucinated, like visit_webpage() or open_link()
  • When tools do run, it often fails to extract or parse the data correctly
  • Switching models has not improved reliability. The failures just change format, such as timeouts, bad extractions, or inconsistent step outputs

At this point, I’m also starting to question whether I should continue the course. I want to feel like the things I’m learning will be usable after the course ends, and right now it’s hard to tell what is actually portable versus what only works in this tightly coupled lesson environment. If the rest of the course depends on the same underlying architecture, I’m worried about running into more of the same issues.

That said, I’m planning to check out the LlamaIndex section next. I’ve heard it’s more stable, and I’ll circle back to this lesson later if things improve.

If anyone has managed to get this lesson working in a reliable way, especially with usable song or playlist output, I’d really appreciate hearing how you approached it.

Hopefully this helps save someone else some time if they run into similar issues. Thank you again to the Hugging Face team for building the course and for supporting discussion around it.

1 Like