Production OpenRouter Basics
9 examples to get you started with Production OpenRouter - 6 basic and 3 intermediate.
These sketches use the OpenAI-compatible client pointed at OpenRouter.
Verify model slugs, headers, and fallback fields against current OpenRouter docs at build time.
Prerequisites
- An OpenRouter API key in the environment (
OPENROUTER_API_KEY). openaiPython package (or any OpenAI-compatible client).- Optional: skim Architecting an Agent Around a Model-Agnostic Gateway for the separation of loop vs transport.
Basic Examples
1. Point the OpenAI Client at OpenRouter
One base URL and key; model slugs use provider/model form on OpenRouter.
import os
from openai import OpenAI
client = OpenAI(
base_url="https://openrouter.ai/api/v1",
api_key=os.environ["OPENROUTER_API_KEY"],
)
resp = client.chat.completions.create(
model="openai/gpt-4o-mini", # verify slug at build
messages=[{"role": "user", "content": "Say ready in one word."}],
)
print(resp.choices[0].message.content)- Keep
base_urland key construction in one module for the whole agent. - Prefer env vars over committed secrets.
- Treat model slugs as config, not copy-pasted literals in every file.
Related: Architecting an Agent Around a Model-Agnostic Gateway - gateway pattern
2. Add Production Identity Headers
OpenRouter accepts optional app attribution headers for ranking and support context.
client = OpenAI(
base_url="https://openrouter.ai/api/v1",
api_key=os.environ["OPENROUTER_API_KEY"],
default_headers={
"HTTP-Referer": "https://example.com", # your site
"X-Title": "support-agent-prod",
},
)- Use a stable app name per deployment environment.
- Do not put secrets in
X-Titleor referer. - Attribution does not replace auth; the Bearer key still authorizes spend.
Related: OpenRouter in Production Best Practices - ops hygiene
3. Minimal Tool-Using Agent Turn
One model call with tools; execute locally; return observations on the next turn.
tools = [{
"type": "function",
"function": {
"name": "get_order",
"description": "Fetch order status by id",
"parameters": {
"type": "object",
"properties": {"order_id": {"type": "string"}},
"required": ["order_id"],
},
},
}]
def complete(messages):
return client.chat.completions.create(
model="openai/gpt-4o-mini",
messages=messages,
tools=tools,
)- Tool schemas stay in your host; OpenRouter forwards them to capable models.
- Execute tools in your process, never by "trusting" model prose alone.
- Pick models with known tool-calling quality for agent steps.
Related: Tool Calling Through OpenRouter Across Different Providers - multi-provider tools
4. Bound the Agent Loop
Host-enforced max turns beat prompt-only "stop when done."
def run_agent(goal, max_turns=6):
messages = [
{"role": "system", "content": "Use tools when needed. Be concise."},
{"role": "user", "content": goal},
]
for _ in range(max_turns):
msg = complete(messages).choices[0].message
messages.append(msg)
if not msg.tool_calls:
return msg.content
for call in msg.tool_calls:
result = run_tool(call) # your dispatcher
messages.append({
"role": "tool",
"tool_call_id": call.id,
"content": result,
})
return {"status": "max_turns"}- Always return a structured failure when the cap hits.
- Add wall-clock timeout and cost budget for production.
- Log turn index and stop reason on every run.
Related: OpenRouter Rate Limits, Retries, and Backoff Strategies - HTTP resilience
5. Enable Model Fallback on the Request
OpenRouter can try alternate models when the primary fails (verify field names at build).
resp = client.chat.completions.create(
model="anthropic/claude-sonnet-4",
extra_body={
"models": [
"anthropic/claude-sonnet-4",
"openai/gpt-4o",
"google/gemini-2.5-pro",
],
},
messages=messages,
tools=tools,
)- Keep fallbacks capability-compatible (all must support your tools if you pass tools).
- Log which model actually served the response when the API returns that metadata.
- Fallbacks are not a substitute for application retries with backoff.
Related: Building a Self-Hosted Fallback Layer: Ollama + OpenRouter - local + cloud
6. Map Logical Roles to Model Slugs
Config drives routing; the loop only knows role names.
MODEL_REGISTRY = {
"default": "openai/gpt-4o-mini",
"planner": "anthropic/claude-sonnet-4",
"fallback": "google/gemini-2.5-flash",
}
def model_for(role: str) -> str:
return MODEL_REGISTRY[role]- Change production models without editing tool code.
- Use different registries per environment (
dev,staging,prod). - Eval each role's model on a fixed suite before promoting config.
Related: A/B Testing Models in Production via OpenRouter - traffic splits
Intermediate Examples
7. Retry Transient Errors with Backoff
Handle 429 and 5xx at the client boundary.
import time
from openai import RateLimitError, APIStatusError
def complete_with_retry(messages, model, attempts=4):
delay = 0.5
for i in range(attempts):
try:
return client.chat.completions.create(
model=model, messages=messages, tools=tools,
)
except (RateLimitError, APIStatusError) as e:
if i == attempts - 1:
raise
time.sleep(delay)
delay = min(delay * 2, 8.0)- Retry only idempotent model calls; do not blindly re-run side-effecting tools.
- Cap total attempts and total wait time.
- Prefer jitter in high-concurrency services.
Related: OpenRouter Rate Limits, Retries, and Backoff Strategies - full strategy
8. Log Usage per Turn
Production needs token and latency signals, not only final answers.
import time
def complete_logged(messages, model, run_id):
t0 = time.perf_counter()
resp = client.chat.completions.create(model=model, messages=messages)
ms = (time.perf_counter() - t0) * 1000
u = resp.usage
print({
"run_id": run_id,
"model": model,
"latency_ms": round(ms, 1),
"prompt_tokens": getattr(u, "prompt_tokens", None),
"completion_tokens": getattr(u, "completion_tokens", None),
})
return resp- Correlate logs with your agent
run_id/ trace id. - Aggregate cost by model role weekly.
- Redact user content in default logs.
Related: OpenRouter in Production Best Practices - observability
9. Feature-Flag the Model Registry
Ship model changes like code: flag, canary, then full.
def resolve_model(role: str, flags: dict) -> str:
if flags.get("use_sonnet_planner") and role == "planner":
return "anthropic/claude-sonnet-4"
return MODEL_REGISTRY[role]- Roll forward or back without redeploying tools when config is externalized.
- Pair flags with eval gates for quality regressions.
- Document the default path for on-call.
Related: Case Study: Migrating a Single-Provider Agent to OpenRouter - migration path
Related
- Architecting an Agent Around a Model-Agnostic Gateway - design pattern
- Tool Calling Through OpenRouter Across Different Providers - tools recipe
- Combining OpenRouter with LangChain, CrewAI, and LlamaIndex - frameworks
- OpenRouter Rate Limits, Retries, and Backoff Strategies - 429 handling
- OpenRouter in Production Best Practices - checklist
Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana
@solana/web3.js+@solana/spl-token.