Handling Tool Errors Gracefully Inside an Agent Loop

When a tool fails, the worst default is crashing the worker or swallowing the error into an empty string.

The durable pattern is: catch at the host boundary, convert the failure into a structured tool observation, enforce a retry budget, and let the model adapt - or stop with a clear reason when budgets expire.

Summary

Treat tool failures as first-class observations (ok: false, typed error, retryable flag, short message). Validate before execute, never omit a tool result for a proposed call id, cap identical retries in the host, and escalate when the model cannot make progress.

Recipe

1. Wrap every tool execution in host-side try/except with timeouts.
2. Normalize successes and failures into one JSON envelope.
3. Classify errors: invalid_args, not_found, timeout, rate_limited, auth, upstream, unknown_tool.
4. Mark retryable honestly; auth misconfig is rarely retryable mid-run.
5. Always append a tool message for each call id, including failures.
6. Enforce host retry budgets (per call signature and per run), not infinite model hope.
7. On retryable failures, optionally sleep/jitter once in the host before re-inviting the model.
8. Strip secrets from error messages before they enter context or logs.
9. Detect no-progress loops (same tool + same args + same error) and stop or escalate.
10. Surface stop_reason and error metrics to ops; keep user-facing copy calm and non-leaky.

Working Example

import json
import time
from dataclasses import dataclass
from typing import Any, Callable
 
@dataclass
class ToolObservation:
    ok: bool
    data: Any = None
    error_type: str | None = None
    retryable: bool = False
    message: str | None = None
 
    def to_content(self) -> str:
        return json.dumps(
            {
                "ok": self.ok,
                "data": self.data,
                "error_type": self.error_type,
                "retryable": self.retryable,
                "message": self.message,
            }
        )
 
class RetryBudget:
    def __init__(self, max_per_signature: int = 2, max_tool_errors: int = 5):
        self.max_per_signature = max_per_signature
        self.max_tool_errors = max_tool_errors
        self.sig_counts: dict[str, int] = {}
        self.total_errors = 0
 
    def allow(self, signature: str) -> bool:
        if self.total_errors >= self.max_tool_errors:
            return False
        return self.sig_counts.get(signature, 0) < self.max_per_signature
 
    def record_error(self, signature: str) -> None:
        self.sig_counts[signature] = self.sig_counts.get(signature, 0) + 1
        self.total_errors += 1
 
def run_tool(
    name: str,
    args: dict,
    impl: Callable[..., Any],
    budget: RetryBudget,
) -> ToolObservation:
    signature = f"{name}:{json.dumps(args, sort_keys=True)}"
    if name == "get_order" and "order_id" not in args:
        return ToolObservation(
            ok=False,
            error_type="invalid_args",
            retryable=True,
            message="Missing required order_id (integer).",
        )
    if not budget.allow(signature):
        return ToolObservation(
            ok=False,
            error_type="retry_budget_exhausted",
            retryable=False,
            message="Host blocked another retry of this identical call.",
        )
    try:
        data = impl(**args)
        return ToolObservation(ok=True, data=data)
    except TimeoutError:
        budget.record_error(signature)
        return ToolObservation(
            ok=False,
            error_type="timeout",
            retryable=True,
            message="Upstream timed out after 10s; try once more or simplify the request.",
        )
    except KeyError as exc:
        budget.record_error(signature)
        return ToolObservation(
            ok=False,
            error_type="not_found",
            retryable=False,
            message=f"Resource not found: {exc}",
        )
    except Exception as exc:  # narrow per tool in production
        budget.record_error(signature)
        return ToolObservation(
            ok=False,
            error_type="upstream",
            retryable=False,
            message=f"{type(exc).__name__}: {str(exc)[:200]}",
        )
 
# After a model tool_call:
# obs = run_tool(name, args, REGISTRY[name], budget)
# messages.append({"role": "tool", "tool_call_id": call_id, "content": obs.to_content()})
# if obs.error_type == "retry_budget_exhausted": stop_reason = "tool_error_budget"

Deep Dive

Errors are observations

The model cannot adapt to a failure it never sees. An empty tool result looks like success with no data, which invites hallucination.

Always return a parseable object with ok and a short, actionable message.

Validate before side effects

def preflight(name: str, args: dict) -> ToolObservation | None:
    if name not in REGISTRY:
        return ToolObservation(
            ok=False,
            error_type="unknown_tool",
            retryable=False,
            message=f"Unknown tool {name}. Available: {sorted(REGISTRY)}",
        )
    # jsonschema.validate(args, SCHEMAS[name]) → invalid_args
    return None

Catching bad args before writes prevents partial mutations and useless upstream load.

Classification drives policy

Host retries vs model retries

• Host soft-retry: transparent second attempt on pure transport blips before observing failure (use sparingly, log both attempts).
• Model retry: model sees the error and chooses new args or another tool.
• Hard stop: budgets exhausted, auth broken, or non-progress detected.

Do not double-count blindly. If the host already retried twice, the observation should say so.

Identical signature detection

def is_no_progress(history: list[tuple[str, str, str]]) -> bool:
    # history entries: (name, args_json, error_type)
    return len(history) >= 3 and len(set(history[-3:])) == 1

Three identical failures in a row is a loop smell. Stop with stop_reason=no_progress rather than spending another frontier-model turn.

Partial batches

When parallel tool calls partially fail, return success and error observations for each id. See Parallel Tool Calls.

User-facing vs model-facing text

Framework hooks

LangGraph, OpenAI Agents SDK, CrewAI, Pydantic AI, and similar stacks offer tool error hooks or exception-to-result helpers. Whatever the framework, keep the envelope consistent so evals and traces stay comparable.

Gotchas

1. Raising through the loop. Uncaught exceptions drop trajectories and confuse orchestrators.
2. Lying retryable: true on auth errors. Burns budget and can lock accounts.
3. Huge stack traces in context. Costly and sometimes sensitive; truncate.
4. Silent host retries without logging. Looks like model latency; hides upstream pain.
5. Empty catch blocks. Worst case: model invents a successful world.
6. No global error budget. Per-tool caps alone still allow death-by-many-tools.
7. Treating not_found as transport failure. Causes useless identical retries.

Alternatives

FAQs

Related

• How Function Calling Actually Works Under the Hood - where observations re-enter
• Tool Use & Function Calling Basics - dispatch basics
• JSON Schema for Tool Parameters: A Reference Guide - preventing invalid_args
• Parallel Tool Calls: When and How Models Call Multiple Tools at Once - partial batch failures
• Tool Use & Function Calling Best Practices - broader habits
• Fallback Chains: Degrading Gracefully When a Model or Tool Fails - higher-level degrade paths
• Circuit Breakers for Agent Tool Calls - systemic outages
• Stopping Conditions: How an Agent Knows When It's Done - exit reasons

Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana @solana/web3.js + @solana/spl-token.