Timeout Strategies for Long-Running Agent Steps

A timeout strategy sets hard clocks on model calls, tool calls, and whole runs so an agent cannot hang forever.

Long steps are normal in agent systems; unbounded steps are incidents waiting to happen.

Summary

Apply nested deadlines (run > turn > tool > HTTP), fail with structured timeout observations, cancel work when possible, and leave enough budget for fallbacks instead of spending the whole SLO on one hung call.

Recipe

1. Define three clocks: run wall time, per-turn/step, per-tool/provider call.
2. Prefer native client deadlines (HTTP timeout, SDK max duration) over hope.
3. On timeout, return a typed observation (error_type=timeout) and count it toward budgets.
4. Attempt cancellation of underlying tasks, subprocesses, or streaming reads.
5. Reserve remaining run budget for degrade/fallback; do not start a long tool with 200ms left.
6. Align timeouts with user-facing SLOs and with upstream SLAs.
7. Log duration histograms per tool and model; retune outliers.
8. Test hung dependencies so the loop always terminates.

Working Example

import time
from concurrent.futures import ThreadPoolExecutor, TimeoutError as FuturesTimeout
from dataclasses import dataclass
from typing import Any, Callable, TypeVar
 
T = TypeVar("T")
 
 
@dataclass
class Deadline:
    """Absolute deadline in time.time() seconds."""
 
    at: float
 
    @classmethod
    def in_seconds(cls, seconds: float) -> "Deadline":
        return cls(at=time.time() + seconds)
 
    def remaining(self) -> float:
        return max(0.0, self.at - time.time())
 
    def expired(self) -> bool:
        return self.remaining() <= 0
 
 
@dataclass
class Observation:
    ok: bool
    data: Any = None
    error_type: str | None = None
    message: str | None = None
 
 
def call_with_timeout(fn: Callable[[], T], timeout_s: float) -> Observation:
    if timeout_s <= 0:
        return Observation(ok=False, error_type="timeout", message="no budget left")
    with ThreadPoolExecutor(max_workers=1) as pool:
        fut = pool.submit(fn)
        try:
            return Observation(ok=True, data=fut.result(timeout=timeout_s))
        except FuturesTimeout:
            return Observation(
                ok=False,
                error_type="timeout",
                message=f"exceeded {timeout_s:.2f}s",
            )
 
 
def run_turn(tools: list[tuple[str, Callable[[], Any]]], run_deadline: Deadline) -> list[Observation]:
    results: list[Observation] = []
    for name, fn in tools:
        # Cap each tool, but never exceed remaining run budget
        tool_cap = 2.0
        budget = min(tool_cap, run_deadline.remaining())
        obs = call_with_timeout(fn, budget)
        results.append(obs)
        if not obs.ok and obs.error_type == "timeout":
            # stop scheduling more tools this turn
            break
        if run_deadline.expired():
            break
    return results
 
 
def slow() -> str:
    time.sleep(3)
    return "late"
 
 
def fast() -> str:
    return "ok"
 
 
deadline = Deadline.in_seconds(2.5)
print(run_turn([("slow", slow), ("fast", fast)], deadline))

Production code should use asyncio cancel scopes, HTTP timeout= parameters, or worker job deadlines instead of only thread result timeouts.

Deep Dive

Nested budgets beat a single global sleep

Each inner timeout must be less than the remaining outer budget.

Model calls need timeouts too

Providers hang. Streams stall mid-token.

Set:

• Connect timeout
• Read / idle timeout between chunks
• Total generation cap when the API supports it

A tool timeout with an infinite model call still freezes the turn.

Cooperative vs preemptive cancellation

• Cooperative: check deadlines between steps (necessary).
• Preemptive: cancel tasks, close HTTP, SIGTERM child processes (necessary for long tools).

ThreadPool result(timeout=) alone may leave the worker thread running. Prefer APIs with real cancel.

Streaming and long research tools

For scrapers, browsers, or multi-page fetches:

• Chunk work into sub-deadlines.
• Checkpoint partial results into durable state.
• Return partial observation on timeout rather than nothing when partial is safe.

Partial search hits beat an empty hang.

Aligning with SLOs

If p95 time-to-final SLO is 45s:

• Run deadline might be 45-60s.
• Single tool might be 8-15s.
• Fallbacks need leftover budget (for example, never start primary tool with <2s left if fallback needs 3s).

See Setting SLOs for Agent Latency, Success Rate, and Cost.

Framework notes

LangGraph and similar systems expose per-node or run-level time controls in recent versions (verify at build).

Whatever the framework:

• Put deadlines in shared config.
• Enforce in one client layer used by all tools.
• Do not scatter magic numbers inside prompts.

Timeouts and retries

A timeout can be retryable once with backoff if the budget allows.

Never:

• Retry a 30s timeout three times under a 40s run deadline.
• Hide timeouts from the model when argument changes might help.

User-facing behavior

On run timeout:

• Persist terminal status timed_out.
• Show what completed.
• Offer resume or escalate rather than auto-restarting an identical long plan blindly.

Gotchas

1. Only max turns, no wall clock. Slow tools make "3 turns" last an hour.
2. Tool timeout > run timeout. Unreachable code paths and confusing races.
3. No cancel on timeout. Cost and locks continue after UX failed.
4. Identical timeout for every tool. Search and bulk export need different caps.
5. Counting wall time while waiting on human approval. Freeze the run clock or use a separate approval SLA.
6. Client timeout longer than load balancer idle timeout. LB cuts first; harder errors.
7. Swallowing TimeoutError into empty success. Hallucination magnet.

Alternatives

For multi-minute research, prefer async jobs with nested timeouts inside the worker.

FAQs

Related

• Reliability Engineering Basics - first timeout sketch
• Setting SLOs for Agent Latency, Success Rate, and Cost - what to bound toward
• Circuit Breakers for Agent Tool Calls - repeated timeouts as open signal
• Fallback Chains: Degrading Gracefully When a Model or Tool Fails - use remaining budget
• Why Agents Fail Differently Than Traditional Services - hang modes
• Kill Switches: Stopping a Runaway Agent Mid-Execution - emergency stop
• LangGraph 1.0's Per-Node Timeouts and v2 Streaming - framework hooks

Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana @solana/web3.js + @solana/spl-token.