Why agents fail (and compound)
Agents are loops, so a single error compounds: a wrong tool call → a misleading observation → worse reasoning → more wrong calls. A model that's 95% reliable per step is only ~60% reliable over 10 steps. Reliability engineering is the hard part of shipping agents.
The main failure modes
- Infinite / repetitive loops — the agent retries the same failing action. Fix: max-step cap, loop detection (same action+args twice → break), and a "give up → human" path.
- Wrong tool / wrong arguments — vague tool descriptions or free-string params. Fix: better descriptions, enums, input validation, and actionable errors the model can recover from.
- Context bloat / lost-in-the-middle — long trajectories overflow and degrade reasoning. Fix: summarize/truncate old steps, curate tool outputs.
- Cost/latency blowups — each step is an LLM call plus tool latency; sequential tools stack up. Fix: budgets, parallelize independent tools, cache, use a smaller model for simple steps.
- Unsafe actions — an agent with write/delete/spend tools can do real damage, and is vulnerable to prompt injection via tool results (a malicious document telling it to exfiltrate data). Fix: least-privilege tools, human-in-the-loop for irreversible actions, and treat tool outputs as untrusted.
Guardrails
- Bounded loops — max steps, token/cost budget, timeout.
- Human-in-the-loop — require approval for destructive/expensive/irreversible actions (send money, delete, email customers).
- Authorize the user, not the model — every tool's side effect is checked against the user's permissions, never the model's say-so.
- Input/output guardrails — validate tool inputs; sanitize/verify outputs; detect injection in retrieved content.
- Observability — log the full trajectory (every thought, call, result, cost) so you can debug and audit.
Cost discipline
Track tokens and cost per run, set per-user budgets, alert on runaway loops, and always compare the agent against a simpler baseline — if a workflow is 90% as good at 10% of the cost and latency, ship the workflow.