OpenRouter Rate Limits, Retries, and Backoff Strategies

Agents multiply HTTP calls: each turn can mean one or more model requests plus tools.

Without disciplined retries, a single 429 storm becomes a thundering herd, a cost spike, or a stuck loop.

This page is a practical recipe for rate limits and transient errors when OpenRouter is your model gateway.

Summary

Retry idempotent model calls on 429 and selected 5xx with exponential backoff and jitter; cap attempts; prefer model/provider fallback after budgeted retries; never blindly re-execute side-effecting tools.

Recipe

1. Classify errors

Exact codes and bodies can vary - parse carefully and verify against current OpenRouter docs at build.

2. Cap retries and wall time

Example policy: max 4 attempts, max 20s total wait, deadline for the whole agent turn.

3. Use exponential backoff with jitter

sleep = min(cap, base * 2**attempt) * Uniform(0.5, 1.5) (or full jitter variant).

4. Honor Retry-After when present

If the response includes a retry delay header, prefer it over your pure exponential guess (when sane and capped).

5. Separate model retries from tool retries

Tools that charge cards or send email need idempotency keys or "execute once" semantics.

6. Add fallback after N failures

Switch to a secondary OpenRouter model or provider preference instead of infinite retries on one slug.

7. Shed load when saturated

Under global 429 rates, reduce concurrency, queue noncritical jobs, and return degraded responses.

Working Example

import os
import random
import time
from openai import OpenAI, RateLimitError, APIStatusError, APITimeoutError
 
client = OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key=os.environ["OPENROUTER_API_KEY"],
)
 
RETRY_STATUS = {429, 500, 502, 503, 504}
 
def complete_with_backoff(
    *,
    messages,
    model: str,
    fallback_models: list[str] | None = None,
    tools=None,
    max_attempts: int = 4,
    base_delay: float = 0.5,
    max_delay: float = 8.0,
):
    models = [model] + (fallback_models or [])
    last_err = None
    for m in models:
        delay = base_delay
        for attempt in range(max_attempts):
            try:
                return client.chat.completions.create(
                    model=m,
                    messages=messages,
                    tools=tools,
                )
            except RateLimitError as e:
                last_err = e
            except APITimeoutError as e:
                last_err = e
            except APIStatusError as e:
                last_err = e
                if getattr(e, "status_code", None) not in RETRY_STATUS:
                    raise
            if attempt == max_attempts - 1:
                break  # try next model
            sleep_for = min(max_delay, delay) * random.uniform(0.5, 1.5)
            time.sleep(sleep_for)
            delay *= 2
    raise RuntimeError(f"exhausted_retries: {last_err!r}")

Tune limits per environment; production services should also bound total agent wall-clock time outside this helper.

Deep Dive

Why agents hit limits harder than chat UIs

A single user message might trigger 5-15 model calls (plan, act, repair, summarize).

Concurrency across tenants multiplies that further.

Design limits at three layers:

1. Per-run max model calls / max turns
2. Per-tenant concurrency and budget
3. Client retry/fallback policy for OpenRouter HTTP errors

OpenRouter-specific notes

OpenRouter aggregates many providers; limits can come from your OpenRouter account/key, free-model tiers, or upstream provider capacity.

A 429 might mean "slow down this key" or "that model/provider is hot."

Falling back to another model slug often recovers faster than hammering the same one.

Free and heavily discounted models typically have stricter rate limits - do not put them on your hottest agent path without caps.

Spend limits and key credits are related but not identical to RPM limits: a paid key can still 429 under burst traffic.

Verify current limit behavior and headers in OpenRouter documentation at build time.

Backoff shapes

Interaction with OpenRouter model fallbacks

Request-level model lists (when configured) can fail over inside one HTTP call path.

You still want client-side retries for connection drops and some 429s.

Avoid stacking unbounded retries on top of unbounded fallbacks - product of both is your true worst-case cost.

Observability

Log: status code, attempt, sleep_ms, model, fallback index, run_id.

Alert on: rising 429 rate, retry budget exhaustion, fallback share.

Dashboard cost separately for retries so "invisible" duplicate calls show up.

Client concurrency

A thread pool of 100 agents each retrying 4 times will amplify outages.

Use bounded executors, bulkheads per tenant, and optional token buckets before you call OpenRouter.

Gotchas

1. Retrying non-idempotent tool side effects - Separate layers; use idempotency keys.
2. No max delay cap - Exponential growth can sleep for minutes and still fail.
3. Synchronized retries - Without jitter, all workers wake together and re-429.
4. Treating all 400s as transient - Schema bugs need deploys, not sleep.
5. Ignoring credit exhaustion - Looks like hard failure; retry loops burn worker time.
6. Infinite agent loops on model errors - Host max turns must apply even when every call errors.
7. Logging full prompts on every retry - Multiplies PII in log volume.

Alternatives

FAQs

Related

• Production OpenRouter Basics - client setup
• Architecting an Agent Around a Model-Agnostic Gateway - put retries in the gateway
• Building a Self-Hosted Fallback Layer: Ollama + OpenRouter - local shed
• A/B Testing Models in Production via OpenRouter - avoid 429 skew in tests
• OpenRouter in Production Best Practices - full checklist

Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana @solana/web3.js + @solana/spl-token.