Why Agents Fail Differently Than Traditional Services

Traditional services fail in familiar ways: timeouts, 5xx responses, bad deploys, and saturated pools.

Agents add failure modes that look like progress while they burn budget: stuck loops, invented tool arguments, thrashing retries, and partial side effects that leave the world half-changed.

Reliability for agents is therefore about bounding autonomous work, not only keeping a process up.

Summary

Agents combine non-deterministic model choices with multi-step tool loops, so outages and bugs often appear as expensive thrashing rather than a clean crash.
Insight: Classic SLIs (uptime, p99 HTTP latency) miss loop storms, tool hallucination, and cost spikes that destroy trust before the host process dies.
Key Concepts: stuck loops, hallucinated tools/args, cascading retries, partial mutations, observation quality, host-owned bounds.
When to Use: Any production agent with tools, multi-turn reasoning, or paid model calls.
Limitations/Trade-offs: More bounds can cut completion rate; fewer bounds cut bills and sleep. You must measure both reliability and task success.
Related Topics: circuit breakers, fallback chains, timeouts, SLOs, postmortems, kill switches.

Foundations

A traditional request handler is usually:

Accept input.
Call a few dependencies with fixed code paths.
Return success or a typed error.

An agent run is closer to:

Accept a goal.
Let a model choose tools and arguments repeatedly.
Fold observations back into context.
Stop only when policy or luck says so.

That loop multiplies variance. The same user goal can take 2 turns or 40, call different tools, and leave different durable state.

Failure shapes that are rare in fixed services

Failure mode	What you see	Why services rarely do this
Stuck loop	Same plan/tool signature forever	Code paths do not re-plan themselves mid-request
Hallucinated tool call	Unknown name, wrong schema, fake IDs	Hand-written clients call real methods
Cascading retries	Model + host + SDK all retry the same 503	Single stack owns retry policy
Confident wrong success	Final answer claims work done; tools never succeeded	Deterministic code rarely invents completion
Partial multi-step mutation	Email sent, CRM update failed, ticket half-closed	Transactions or single handlers own one unit of work
Context poisoning	Bad tool dump steers every later turn	Stateless handlers do not accumulate garbage mid-request

Uptime is not enough

An agent worker can be "healthy" in Kubernetes while every run is:

Burning tokens on no-progress loops.
Opening tickets with garbage fields.
Retrying a dead dependency on every turn.

Process liveness is necessary. It is not agent reliability.

Mechanics & Interactions

Stuck loops

Loops happen when the host never enforces no-progress rules.

Typical patterns:

Identical tool name + args + error three times in a row.
Oscillating between two tools without new information.
"Try harder" replans that restate the same plan.

The model is optimizing for task completion language, not your cloud bill. Without host caps (max turns, wall clock, signature budgets), thrash is rational from the model's local view.

See also stopping-condition thinking in Timeout Strategies for Long-Running Agent Steps and fundamentals stop rules.

Hallucinated tool calls

Models invent:

Tool names that do not exist.
Required fields that were never in schema.
Plausible but wrong entity IDs.

In a typed microservice, this is a compile-time or client-generation error. In an agent, it is a runtime observation if you handle it well, or a crash/empty result if you do not.

Host duties:

Validate name against a registry.
Validate args against schema before side effects.
Return structured errors the model can repair once or twice.
Stop when the same invalid call repeats.

Cascading retries

Agent stacks often stack retries by accident:

Layer	Typical retry
HTTP client / SDK	2-3 on 429/503
Tool wrapper	Soft retry on timeout
Model loop	New turn calling the same tool
Workflow/orchestrator	Job-level retry of the whole run
User / product UI	"Try again" button

Each layer alone looks reasonable. Together they amplify load during an outage and create retry storms against the dependency you least want to hit.

Circuit breakers and shared budgets are how you collapse those layers into one policy. See Circuit Breakers for Agent Tool Calls.

Observation quality drives the next failure

If the host swallows errors or returns empty strings, the model often hallucinates success and continues with false premises.

If the host dumps huge stack traces, the model may fixate on noise and burn context.

Reliability is coupled to how tool results are shaped. Structured ok / error_type / retryable envelopes matter as much as infrastructure.

Partial side effects

Multi-tool turns and multi-turn plans create distributed-workflow problems:

Tool A commits.
Tool B fails.
Model retries A (duplicate email) or claims overall success.

Traditional services use transactions, sagas, or idempotency keys by design. Agents need the same discipline at the tool boundary, not only in backend services the tools call.

Cost as a reliability signal

For agents, a cost spike is an incident class.

A stuck loop can empty a monthly budget in an afternoon without a single process crash. SLOs and kill switches must include spend and tokens, not only error rate.

Advanced Considerations & Applications

Multi-agent graphs multiply the same modes

Handoffs add:

Privilege expansion if specialists inherit broader tools.
Duplicate work if two agents retry the same dependency.
Harder kill propagation if only the root run is stopped.

Treat the graph id as the reliability unit for kill, budget, and postmortem timelines.

Frameworks do not erase uniqueness

LangGraph, CrewAI, OpenAI Agents SDK, Pydantic AI, and similar runtimes give loops, interrupts, and retries.

They do not invent your no-progress detector, tool circuit breaker, or cost SLO. Host policy still owns those.

Trade-offs

Approach	Strength	Weakness
Treat agents like HTTP microservices only	Familiar ops tooling	Misses loops, hallucination, cost storms
Bound every turn and tool (this section's theme)	Finite blast radius	Needs careful UX when runs stop early
Human approval on every step	High safety	Does not scale; fatigue creates rubber stamps
Unlimited autonomy with good prompts	Fast demos	Production incidents look "creative"

Prefer hard host bounds plus selective human gates over prompt-only caution.

What good failure looks like

A reliable agent failure is:

Terminal with a clear stop_reason (timeout, budget, circuit_open, no_progress, killed).
Observable with traces of tools, models, and decisions.
Bounded in time, money, and side effects.
Recoverable via fallback, degrade, retry of a new run, or human handoff - not silent infinite hope.

Common Misconceptions

"If the model is smart enough, loops will not happen." Capability reduces some thrash; it does not replace caps.
"Retries always improve reliability." Uncoordinated retries worsen outages and duplicate side effects.
"Crash-free workers mean the agent is reliable." Thrashing workers are often crash-free.
"Tool schemas prevent hallucination entirely." They reduce it; hosts must still validate and budget.
"Classic APM is enough." You also need run-level success, loop metrics, and cost SLIs.
"Failing closed is always wrong for UX." A clear degrade beats a wrong refund or spam burst.

FAQs

How is this different from non-deterministic ML inference endpoints?

Single-shot inference has variance in output text. Agents add iterative tool use and durable side effects, so variance becomes multi-step state change and spend.

What is the first metric to add beyond uptime?

Run success rate with an explicit definition of success, plus p95 run latency and cost per successful run.

Do deterministic workflows with an LLM step count as agents for this article?

If there is no open-ended tool loop, many classic reliability patterns apply directly. As soon as the model chooses tools repeatedly, agent-specific modes dominate.

Why do cascading retries appear more in agents?

Multiple autonomous layers (model, tool host, SDK, job runner) each add "helpful" retries without a shared budget.

Is a hallucinated final answer a reliability issue or a quality issue?

Both. If tools failed and the answer claimed success, that is a reliability/integrity failure, not only "bad vibes" quality.

How do kill switches relate to these failure modes?

Kills stop runaway behavior mid-incident. They do not replace design-time bounds; they are the emergency brake when bounds or deploys fail.

Can eval suites catch these failures?

Yes for known thrash patterns and tool-error handling. Production still needs runtime budgets because novel goals create novel loops.

Should we disable model retries entirely?

No. Allow limited, classified retries. Disable unlimited identical retries and stack-wide amplification.

How do partial failures show up in support tickets?

Users report "it said it worked" or "it did it twice." Trace tool timelines, not only the final message.

What should engineers unlearn from pure backend SRE?

That a green dashboard of CPU and 5xx is sufficient. Agent SRE needs loop, tool, and cost semantics.

Are open-source coding agents immune?

No. They often have shell tools with huge blast radius; thrash can mean destructive commands, not only API spend.

Where should we document agent failure modes for the team?

In runbooks and postmortem templates. Start from Postmortems for Agent Incidents: A Template.

Reliability Engineering Basics - first retry wrapper
Circuit Breakers for Agent Tool Calls - stop dependency thrash
Fallback Chains: Degrading Gracefully When a Model or Tool Fails - ordered degrade
Setting SLOs for Agent Latency, Success Rate, and Cost - measurable targets
Timeout Strategies for Long-Running Agent Steps - time bounds
Reliability Engineering Best Practices - checklist
Kill Switches: Stopping a Runaway Agent Mid-Execution - emergency stop
Handling Tool Errors Gracefully Inside an Agent Loop - error observations

Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana @solana/web3.js + @solana/spl-token.

Agents add failure modes that look like progress while they burn budget: stuck loops, invented tool arguments, thrashing retries, and partial side effects that leave the world half-changed.

Reliability for agents is therefore about bounding autonomous work, not only keeping a process up.

Summary

Agents combine non-deterministic model choices with multi-step tool loops, so outages and bugs often appear as expensive thrashing rather than a clean crash.
Insight: Classic SLIs (uptime, p99 HTTP latency) miss loop storms, tool hallucination, and cost spikes that destroy trust before the host process dies.
Key Concepts: stuck loops, hallucinated tools/args, cascading retries, partial mutations, observation quality, host-owned bounds.
When to Use: Any production agent with tools, multi-turn reasoning, or paid model calls.
Limitations/Trade-offs: More bounds can cut completion rate; fewer bounds cut bills and sleep. You must measure both reliability and task success.
Related Topics: circuit breakers, fallback chains, timeouts, SLOs, postmortems, kill switches.

Foundations

A traditional request handler is usually:

Accept input.
Call a few dependencies with fixed code paths.
Return success or a typed error.

An agent run is closer to:

Accept a goal.
Let a model choose tools and arguments repeatedly.
Fold observations back into context.
Stop only when policy or luck says so.

That loop multiplies variance. The same user goal can take 2 turns or 40, call different tools, and leave different durable state.

Failure shapes that are rare in fixed services

Failure mode	What you see	Why services rarely do this
Stuck loop	Same plan/tool signature forever	Code paths do not re-plan themselves mid-request
Hallucinated tool call	Unknown name, wrong schema, fake IDs	Hand-written clients call real methods
Cascading retries	Model + host + SDK all retry the same 503	Single stack owns retry policy
Confident wrong success	Final answer claims work done; tools never succeeded	Deterministic code rarely invents completion
Partial multi-step mutation	Email sent, CRM update failed, ticket half-closed	Transactions or single handlers own one unit of work
Context poisoning	Bad tool dump steers every later turn	Stateless handlers do not accumulate garbage mid-request

Uptime is not enough

An agent worker can be "healthy" in Kubernetes while every run is:

Burning tokens on no-progress loops.
Opening tickets with garbage fields.
Retrying a dead dependency on every turn.

Process liveness is necessary. It is not agent reliability.

Mechanics & Interactions

Stuck loops

Loops happen when the host never enforces no-progress rules.

Typical patterns:

Identical tool name + args + error three times in a row.
Oscillating between two tools without new information.
"Try harder" replans that restate the same plan.

The model is optimizing for task completion language, not your cloud bill. Without host caps (max turns, wall clock, signature budgets), thrash is rational from the model's local view.

See also stopping-condition thinking in Timeout Strategies for Long-Running Agent Steps and fundamentals stop rules.

Hallucinated tool calls

Models invent:

Tool names that do not exist.
Required fields that were never in schema.
Plausible but wrong entity IDs.

In a typed microservice, this is a compile-time or client-generation error. In an agent, it is a runtime observation if you handle it well, or a crash/empty result if you do not.

Host duties:

Validate name against a registry.
Validate args against schema before side effects.
Return structured errors the model can repair once or twice.
Stop when the same invalid call repeats.

Cascading retries

Agent stacks often stack retries by accident:

Layer	Typical retry
HTTP client / SDK	2-3 on 429/503
Tool wrapper	Soft retry on timeout
Model loop	New turn calling the same tool
Workflow/orchestrator	Job-level retry of the whole run
User / product UI	"Try again" button

Each layer alone looks reasonable. Together they amplify load during an outage and create retry storms against the dependency you least want to hit.

Circuit breakers and shared budgets are how you collapse those layers into one policy. See Circuit Breakers for Agent Tool Calls.

Observation quality drives the next failure

If the host swallows errors or returns empty strings, the model often hallucinates success and continues with false premises.

If the host dumps huge stack traces, the model may fixate on noise and burn context.

Reliability is coupled to how tool results are shaped. Structured ok / error_type / retryable envelopes matter as much as infrastructure.

Partial side effects

Multi-tool turns and multi-turn plans create distributed-workflow problems:

Tool A commits.
Tool B fails.
Model retries A (duplicate email) or claims overall success.

Traditional services use transactions, sagas, or idempotency keys by design. Agents need the same discipline at the tool boundary, not only in backend services the tools call.

Cost as a reliability signal

For agents, a cost spike is an incident class.

A stuck loop can empty a monthly budget in an afternoon without a single process crash. SLOs and kill switches must include spend and tokens, not only error rate.

Advanced Considerations & Applications

Multi-agent graphs multiply the same modes

Handoffs add:

Privilege expansion if specialists inherit broader tools.
Duplicate work if two agents retry the same dependency.
Harder kill propagation if only the root run is stopped.

Treat the graph id as the reliability unit for kill, budget, and postmortem timelines.

Frameworks do not erase uniqueness

LangGraph, CrewAI, OpenAI Agents SDK, Pydantic AI, and similar runtimes give loops, interrupts, and retries.

They do not invent your no-progress detector, tool circuit breaker, or cost SLO. Host policy still owns those.

Trade-offs

Approach	Strength	Weakness
Treat agents like HTTP microservices only	Familiar ops tooling	Misses loops, hallucination, cost storms
Bound every turn and tool (this section's theme)	Finite blast radius	Needs careful UX when runs stop early
Human approval on every step	High safety	Does not scale; fatigue creates rubber stamps
Unlimited autonomy with good prompts	Fast demos	Production incidents look "creative"

Prefer hard host bounds plus selective human gates over prompt-only caution.

What good failure looks like

A reliable agent failure is:

Terminal with a clear stop_reason (timeout, budget, circuit_open, no_progress, killed).
Observable with traces of tools, models, and decisions.
Bounded in time, money, and side effects.
Recoverable via fallback, degrade, retry of a new run, or human handoff - not silent infinite hope.

Common Misconceptions

"If the model is smart enough, loops will not happen." Capability reduces some thrash; it does not replace caps.
"Retries always improve reliability." Uncoordinated retries worsen outages and duplicate side effects.
"Crash-free workers mean the agent is reliable." Thrashing workers are often crash-free.
"Tool schemas prevent hallucination entirely." They reduce it; hosts must still validate and budget.
"Classic APM is enough." You also need run-level success, loop metrics, and cost SLIs.
"Failing closed is always wrong for UX." A clear degrade beats a wrong refund or spam burst.

FAQs

How is this different from non-deterministic ML inference endpoints?

Single-shot inference has variance in output text. Agents add iterative tool use and durable side effects, so variance becomes multi-step state change and spend.

What is the first metric to add beyond uptime?

Run success rate with an explicit definition of success, plus p95 run latency and cost per successful run.

Do deterministic workflows with an LLM step count as agents for this article?

If there is no open-ended tool loop, many classic reliability patterns apply directly. As soon as the model chooses tools repeatedly, agent-specific modes dominate.

Why do cascading retries appear more in agents?

Multiple autonomous layers (model, tool host, SDK, job runner) each add "helpful" retries without a shared budget.

Is a hallucinated final answer a reliability issue or a quality issue?

Both. If tools failed and the answer claimed success, that is a reliability/integrity failure, not only "bad vibes" quality.

How do kill switches relate to these failure modes?

Kills stop runaway behavior mid-incident. They do not replace design-time bounds; they are the emergency brake when bounds or deploys fail.

Can eval suites catch these failures?

Yes for known thrash patterns and tool-error handling. Production still needs runtime budgets because novel goals create novel loops.

Should we disable model retries entirely?

No. Allow limited, classified retries. Disable unlimited identical retries and stack-wide amplification.

How do partial failures show up in support tickets?

Users report "it said it worked" or "it did it twice." Trace tool timelines, not only the final message.

What should engineers unlearn from pure backend SRE?

That a green dashboard of CPU and 5xx is sufficient. Agent SRE needs loop, tool, and cost semantics.

Are open-source coding agents immune?

No. They often have shell tools with huge blast radius; thrash can mean destructive commands, not only API spend.

Where should we document agent failure modes for the team?

In runbooks and postmortem templates. Start from Postmortems for Agent Incidents: A Template.

Reliability Engineering Basics - first retry wrapper
Circuit Breakers for Agent Tool Calls - stop dependency thrash
Fallback Chains: Degrading Gracefully When a Model or Tool Fails - ordered degrade
Setting SLOs for Agent Latency, Success Rate, and Cost - measurable targets
Timeout Strategies for Long-Running Agent Steps - time bounds
Reliability Engineering Best Practices - checklist
Kill Switches: Stopping a Runaway Agent Mid-Execution - emergency stop
Handling Tool Errors Gracefully Inside an Agent Loop - error observations

Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana @solana/web3.js + @solana/spl-token.

Why Agents Fail Differently Than Traditional Services

Search Documentation

Why Agents Fail Differently Than Traditional Services