Streaming Responses Through OpenRouter

Streaming is how chat UIs feel fast and how agents show progress.

OpenRouter exposes the same OpenAI-style SSE stream regardless of which upstream provider ultimately serves the model - with a few gateway-specific details you must handle.

Summary

Set stream: true on chat completions against https://openrouter.ai/api/v1.

Consume choices[0].delta.content chunks, ignore SSE comment keep-alives, handle pre-stream HTTP errors differently from mid-stream error events, and remember that canceling a stream does not always stop billing on every provider.

Recipe

Use the OpenAI SDK stream iterator first; drop to raw SSE only when you need custom transport.

import os
from openai import OpenAI
 
client = OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key=os.environ["OPENROUTER_API_KEY"],
)
 
def stream_text(model: str, user: str) -> str:
    parts: list[str] = []
    stream = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": user}],
        stream=True,
    )
    for chunk in stream:
        if not chunk.choices:
            continue
        delta = chunk.choices[0].delta.content or ""
        if delta:
            parts.append(delta)
            print(delta, end="", flush=True)
    print()
    return "".join(parts)

When to reach for this:

• Token-by-token UI updates
• Long agent narrations or chain-of-thought displays (when exposed)
• Cancelling user-aborted generations early on supported providers
• Uniform streaming across multi-provider routes

Working Example

Python OpenAI SDK

import os
from openai import OpenAI
 
client = OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key=os.environ["OPENROUTER_API_KEY"],
)
 
stream = client.chat.completions.create(
    model="openai/gpt-4o-mini",  # verify at build
    messages=[{"role": "user", "content": "Write three bullet agent tips."}],
    stream=True,
)
 
for chunk in stream:
    if getattr(chunk, "error", None):
        raise RuntimeError(chunk.error)
    if not chunk.choices:
        continue
    piece = chunk.choices[0].delta.content or ""
    print(piece, end="", flush=True)

TypeScript OpenAI SDK

import OpenAI from "openai";
 
const client = new OpenAI({
  baseURL: "https://openrouter.ai/api/v1",
  apiKey: process.env.OPENROUTER_API_KEY,
});
 
const stream = await client.chat.completions.create({
  model: "openai/gpt-4o-mini", // verify at build
  messages: [{ role: "user", content: "Write three bullet agent tips." }],
  stream: true,
});
 
for await (const chunk of stream) {
  const piece = chunk.choices[0]?.delta?.content ?? "";
  if (piece) process.stdout.write(piece);
}

TypeScript raw fetch SSE (comment-safe)

const res = await fetch("https://openrouter.ai/api/v1/chat/completions", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.OPENROUTER_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "openai/gpt-4o-mini", // verify at build
    messages: [{ role: "user", content: "Hello" }],
    stream: true,
  }),
});
 
if (!res.ok) {
  console.error(await res.json());
  throw new Error(`HTTP ${res.status}`);
}
 
const reader = res.body!.getReader();
const decoder = new TextDecoder();
let buffer = "";
 
while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });
  for (;;) {
    const idx = buffer.indexOf("\n");
    if (idx === -1) break;
    const line = buffer.slice(0, idx).trim();
    buffer = buffer.slice(idx + 1);
    if (!line || line.startsWith(":")) continue; // keep-alives like ": OPENROUTER PROCESSING"
    if (!line.startsWith("data: ")) continue;
    const data = line.slice(6);
    if (data === "[DONE]") break;
    const parsed = JSON.parse(data);
    if (parsed.error) throw new Error(parsed.error.message);
    const content = parsed.choices?.[0]?.delta?.content;
    if (content) process.stdout.write(content);
  }
}

What this demonstrates:

• Identical streaming API across providers via OpenRouter
• SDK convenience vs raw SSE
• Skipping colon-prefixed SSE comments that are not JSON
• Pre-stream HTTP error checks before reading the body

Deep Dive

SSE shape

Successful streams send data: {chunk} lines and end with data: [DONE].

OpenRouter may insert comment lines such as:

: OPENROUTER PROCESSING

These keep connections alive.

If you JSON.parse them, your loop crashes - skip lines starting with :.

Error timing

Mid-stream error chunks include a top-level error and often finish_reason: "error".

Cancellation and billing

Aborting the client connection can stop generation and billing on many providers.

Some providers do not currently stop billing on abort (OpenRouter documents unsupported sets that have included Bedrock, Groq, Google, Mistral, and others - verify the live list at build).

Product implication: "user hit stop" is not a universal cost-cancel guarantee.

Usage on streams

Final chunks may include usage depending on SDK/API options.

If you need reliable token counts, confirm streaming usage settings for your API skin at build or fetch generation metadata after the fact.

Tool calls while streaming

Tool-call arguments may arrive as partial deltas.

Buffer function name and argument JSON until finish_reason indicates completion, then parse once.

Re-test per model tier - streaming tool payloads are a common integration bug.

Generation correlation

OpenRouter may return X-Generation-Id on responses (verify at build).

Log it next to your request id for support and replay.

Gotchas

• Parsing SSE comments as JSON. Random crashes mid-stream. Fix: ignore : lines or use a compliant parser.
• Assuming mid-stream failover. Partial text already shown cannot be rewound invisibly. Fix: UX for partial failure.
• Not checking res.ok before reading SSE. Error JSON misread as stream. Fix: branch on status first.
• Cancel everywhere expecting free stops. Some providers still bill. Fix: design timeouts and max tokens.
• Buffering the entire stream in a serverless function without flushing. UX stalls. Fix: pipe chunks to the client.
• Mixing stream consumers that each expect exclusive body reads. Fix: one reader or tee explicitly.
• Ignoring finish_reason. Miss length stops vs errors. Fix: record finish reasons in traces.

Alternatives

FAQs

Related

• Calling OpenRouter from Python and TypeScript SDKs - non-stream and stream side by side
• OpenRouter Setup & Routing Basics - first stream example
• Troubleshooting Common OpenRouter Setup Errors - stream failure checklist
• How Model Routing Actually Works Under an OpenRouter Request - failover vs streaming
• OpenRouter Setup & Routing Best Practices - production stream habits

Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana @solana/web3.js + @solana/spl-token.