I’ve spent the last year and a half moving projects from “chatbot with a system prompt” to actual agentic systems — ones that plan, call tools, and complete multi-step tasks without a human babysitting every turn. Most tutorials make this look like a weekend project. It isn’t. Here’s the architecture and the failure modes nobody warns you about.
What “agentic” actually requires
Strip away the hype and an agent is four components in a loop:
- A reasoning core — the LLM itself. We’ve used both hosted APIs and open models (Llama 3.x and Qwen 2.5 fine-tunes served via TGI/vLLM). For tool-heavy workloads, a smaller model with good function-calling accuracy beats a bigger model that hallucinates arguments. Benchmark on your tool schemas, not leaderboards.
- Tool interface layer — where 70% of real engineering lives. Every tool needs a strict schema, input validation, timeouts, and a machine-readable error format the model can recover from. Our biggest early mistake: returning raw stack traces as tool output. The model would “apologize” and retry the identical broken call forever. Now every tool returns structured errors with a
retryableflag and a hint field, and loop behavior improved dramatically. - Planning + control flow — we started with free-form ReAct-style loops. They work in demos and drift in production. What stabilized things: an explicit state machine around the model. The LLM proposes actions; deterministic code decides what’s allowed from the current state. Think of the model as a proposal engine, not the executor. Max-iteration caps and budget ceilings are non-negotiable — an agent that can loop is an agent that will loop.
- Memory — three separate problems people conflate: (a) in-context working memory (summarize aggressively — after ~15 tool calls, transcripts poison the context), (b) episodic memory across sessions (we use embedding retrieval over past task outcomes), (c) long-term structured state (just use a database; not everything needs vectors).
The eval problem
This is where most agent projects quietly die. Single-turn evals tell you almost nothing about multi-step behavior. What we landed on:
- Trajectory evals: record full action sequences on a fixed task suite, score terminal state (“did the refund actually get issued?”), not the politeness of the final message.
- Regression traps: every production failure becomes a permanent test case.
- LLM-as-judge for intermediate steps, but only after we calibrated the judge against human labels — uncalibrated judges gave us confident garbage.
Rough numbers from our experience: a task with 90% per-step reliability across 5 chained steps lands around 59% end-to-end. That math is the whole reason agent architecture matters. You claw reliability back through validation layers, retries with modified prompts, and checkpoints where a human can intervene — not through prompt wizardry.
Things I’d tell past-me
- Start with one narrow workflow, fully instrumented, before generalizing. “General-purpose agent” as v1 is how projects die.
- Deterministic guardrails > prompt instructions. If an action is dangerous, gate it in code. “Please don’t delete records” in a system prompt is a wish, not a control.
- Log every model call with full context. Agent debugging without traces is archaeology.
- Human-in-the-loop isn’t a compromise — for anything irreversible (payments, data mutation, external comms), approval gates are what make deployment survivable.
- Fine-tuning helped us far less than better tool design. We assumed we’d need custom models; mostly we needed better schemas and error messages.
Where I’m still unsure
Multi-agent setups. We’ve experimented with planner/executor splits and reviewer agents, and honestly the added coordination overhead ate most of the gains for our use cases. Single well-instrumented agent + good tools has beaten our multi-agent prototypes almost every time. But I see strong claims in the other direction from people doing research-style workloads.
So, question for this community: for those running agents in production — has anyone found a multi-agent architecture that consistently beats a single-agent loop on reliability, not just on benchmark tasks? And what’s your eval setup for multi-step trajectories? Genuinely curious what’s working outside my bubble.