Why Agents Fail Differently Than Traditional Services
Traditional services fail in familiar ways: timeouts, 5xx responses, bad deploys, and saturated pools.
Agents add failure modes that look like progress while they burn budget: stuck loops, invented tool arguments, thrashing retries, and partial side effects that leave the world half-changed.
Reliability for agents is therefore about bounding autonomous work, not only keeping a process up.
Summary
- Agents combine non-deterministic model choices with multi-step tool loops, so outages and bugs often appear as expensive thrashing rather than a clean crash.
- Insight: Classic SLIs (uptime, p99 HTTP latency) miss loop storms, tool hallucination, and cost spikes that destroy trust before the host process dies.
- Key Concepts: stuck loops, hallucinated tools/args, cascading retries, partial mutations, observation quality, host-owned bounds.
- When to Use: Any production agent with tools, multi-turn reasoning, or paid model calls.
- Limitations/Trade-offs: More bounds can cut completion rate; fewer bounds cut bills and sleep. You must measure both reliability and task success.
- Related Topics: circuit breakers, fallback chains, timeouts, SLOs, postmortems, kill switches.
Foundations
A traditional request handler is usually:
- Accept input.
- Call a few dependencies with fixed code paths.
- Return success or a typed error.
An agent run is closer to:
- Accept a goal.
- Let a model choose tools and arguments repeatedly.
- Fold observations back into context.
- Stop only when policy or luck says so.
That loop multiplies variance. The same user goal can take 2 turns or 40, call different tools, and leave different durable state.
Failure shapes that are rare in fixed services
| Failure mode | What you see | Why services rarely do this |
|---|---|---|
| Stuck loop | Same plan/tool signature forever | Code paths do not re-plan themselves mid-request |
| Hallucinated tool call | Unknown name, wrong schema, fake IDs | Hand-written clients call real methods |
| Cascading retries | Model + host + SDK all retry the same 503 | Single stack owns retry policy |
| Confident wrong success | Final answer claims work done; tools never succeeded | Deterministic code rarely invents completion |
| Partial multi-step mutation | Email sent, CRM update failed, ticket half-closed | Transactions or single handlers own one unit of work |
| Context poisoning | Bad tool dump steers every later turn | Stateless handlers do not accumulate garbage mid-request |
Uptime is not enough
An agent worker can be "healthy" in Kubernetes while every run is:
- Burning tokens on no-progress loops.
- Opening tickets with garbage fields.
- Retrying a dead dependency on every turn.
Process liveness is necessary. It is not agent reliability.
Mechanics & Interactions
Stuck loops
Loops happen when the host never enforces no-progress rules.
Typical patterns:
- Identical tool name + args + error three times in a row.
- Oscillating between two tools without new information.
- "Try harder" replans that restate the same plan.
The model is optimizing for task completion language, not your cloud bill. Without host caps (max turns, wall clock, signature budgets), thrash is rational from the model's local view.
See also stopping-condition thinking in Timeout Strategies for Long-Running Agent Steps and fundamentals stop rules.
Hallucinated tool calls
Models invent:
- Tool names that do not exist.
- Required fields that were never in schema.
- Plausible but wrong entity IDs.
In a typed microservice, this is a compile-time or client-generation error. In an agent, it is a runtime observation if you handle it well, or a crash/empty result if you do not.
Host duties:
- Validate name against a registry.
- Validate args against schema before side effects.
- Return structured errors the model can repair once or twice.
- Stop when the same invalid call repeats.
Cascading retries
Agent stacks often stack retries by accident:
| Layer | Typical retry |
|---|---|
| HTTP client / SDK | 2-3 on 429/503 |
| Tool wrapper | Soft retry on timeout |
| Model loop | New turn calling the same tool |
| Workflow/orchestrator | Job-level retry of the whole run |
| User / product UI | "Try again" button |
Each layer alone looks reasonable. Together they amplify load during an outage and create retry storms against the dependency you least want to hit.
Circuit breakers and shared budgets are how you collapse those layers into one policy. See Circuit Breakers for Agent Tool Calls.
Observation quality drives the next failure
If the host swallows errors or returns empty strings, the model often hallucinates success and continues with false premises.
If the host dumps huge stack traces, the model may fixate on noise and burn context.
Reliability is coupled to how tool results are shaped. Structured ok / error_type / retryable envelopes matter as much as infrastructure.
Partial side effects
Multi-tool turns and multi-turn plans create distributed-workflow problems:
- Tool A commits.
- Tool B fails.
- Model retries A (duplicate email) or claims overall success.
Traditional services use transactions, sagas, or idempotency keys by design. Agents need the same discipline at the tool boundary, not only in backend services the tools call.
Cost as a reliability signal
For agents, a cost spike is an incident class.
A stuck loop can empty a monthly budget in an afternoon without a single process crash. SLOs and kill switches must include spend and tokens, not only error rate.
Advanced Considerations & Applications
Multi-agent graphs multiply the same modes
Handoffs add:
- Privilege expansion if specialists inherit broader tools.
- Duplicate work if two agents retry the same dependency.
- Harder kill propagation if only the root run is stopped.
Treat the graph id as the reliability unit for kill, budget, and postmortem timelines.
Frameworks do not erase uniqueness
LangGraph, CrewAI, OpenAI Agents SDK, Pydantic AI, and similar runtimes give loops, interrupts, and retries.
They do not invent your no-progress detector, tool circuit breaker, or cost SLO. Host policy still owns those.
Trade-offs
| Approach | Strength | Weakness |
|---|---|---|
| Treat agents like HTTP microservices only | Familiar ops tooling | Misses loops, hallucination, cost storms |
| Bound every turn and tool (this section's theme) | Finite blast radius | Needs careful UX when runs stop early |
| Human approval on every step | High safety | Does not scale; fatigue creates rubber stamps |
| Unlimited autonomy with good prompts | Fast demos | Production incidents look "creative" |
Prefer hard host bounds plus selective human gates over prompt-only caution.
What good failure looks like
A reliable agent failure is:
- Terminal with a clear
stop_reason(timeout, budget, circuit_open, no_progress, killed). - Observable with traces of tools, models, and decisions.
- Bounded in time, money, and side effects.
- Recoverable via fallback, degrade, retry of a new run, or human handoff - not silent infinite hope.
Common Misconceptions
- "If the model is smart enough, loops will not happen." Capability reduces some thrash; it does not replace caps.
- "Retries always improve reliability." Uncoordinated retries worsen outages and duplicate side effects.
- "Crash-free workers mean the agent is reliable." Thrashing workers are often crash-free.
- "Tool schemas prevent hallucination entirely." They reduce it; hosts must still validate and budget.
- "Classic APM is enough." You also need run-level success, loop metrics, and cost SLIs.
- "Failing closed is always wrong for UX." A clear degrade beats a wrong refund or spam burst.
FAQs
How is this different from non-deterministic ML inference endpoints?
Single-shot inference has variance in output text. Agents add iterative tool use and durable side effects, so variance becomes multi-step state change and spend.
What is the first metric to add beyond uptime?
Run success rate with an explicit definition of success, plus p95 run latency and cost per successful run.
Do deterministic workflows with an LLM step count as agents for this article?
If there is no open-ended tool loop, many classic reliability patterns apply directly. As soon as the model chooses tools repeatedly, agent-specific modes dominate.
Why do cascading retries appear more in agents?
Multiple autonomous layers (model, tool host, SDK, job runner) each add "helpful" retries without a shared budget.
Is a hallucinated final answer a reliability issue or a quality issue?
Both. If tools failed and the answer claimed success, that is a reliability/integrity failure, not only "bad vibes" quality.
How do kill switches relate to these failure modes?
Kills stop runaway behavior mid-incident. They do not replace design-time bounds; they are the emergency brake when bounds or deploys fail.
Can eval suites catch these failures?
Yes for known thrash patterns and tool-error handling. Production still needs runtime budgets because novel goals create novel loops.
Should we disable model retries entirely?
No. Allow limited, classified retries. Disable unlimited identical retries and stack-wide amplification.
How do partial failures show up in support tickets?
Users report "it said it worked" or "it did it twice." Trace tool timelines, not only the final message.
What should engineers unlearn from pure backend SRE?
That a green dashboard of CPU and 5xx is sufficient. Agent SRE needs loop, tool, and cost semantics.
Are open-source coding agents immune?
No. They often have shell tools with huge blast radius; thrash can mean destructive commands, not only API spend.
Where should we document agent failure modes for the team?
In runbooks and postmortem templates. Start from Postmortems for Agent Incidents: A Template.
Related
- Reliability Engineering Basics - first retry wrapper
- Circuit Breakers for Agent Tool Calls - stop dependency thrash
- Fallback Chains: Degrading Gracefully When a Model or Tool Fails - ordered degrade
- Setting SLOs for Agent Latency, Success Rate, and Cost - measurable targets
- Timeout Strategies for Long-Running Agent Steps - time bounds
- Reliability Engineering Best Practices - checklist
- Kill Switches: Stopping a Runaway Agent Mid-Execution - emergency stop
- Handling Tool Errors Gracefully Inside an Agent Loop - error observations
Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana
@solana/web3.js+@solana/spl-token.