Combining OpenRouter with LangChain, CrewAI, and LlamaIndex

Most agent frameworks already speak OpenAI-compatible chat APIs.

OpenRouter exposes that shape at https://openrouter.ai/api/v1, so you can keep LangChain, CrewAI, or LlamaIndex orchestration and swap only the model backend.

This page gives working wiring patterns and the production rules that stop "works in the tutorial" setups from leaking keys or hardcoding SKUs.

Summary

Construct each framework's chat model once through a shared factory: OpenRouter base URL, API key, model slug, and optional headers.

Do not configure OpenRouter differently in every agent class.

Recipe

1. Centralize env and registry

export OPENROUTER_API_KEY=sk-or-...
export OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
export AGENT_MODEL_DEFAULT=openai/gpt-4o-mini

Map logical roles (default, planner, researcher) to slugs in one module.

2. Build models only in a factory

Every framework entrypoint calls get_chat_model(role) rather than constructing clients inline.

3. Prefer OpenAI-compatible adapters

LangChain ChatOpenAI, LlamaIndex OpenAI-compatible LLMs, and CrewAI/LiteLLM OpenRouter routes all work from the same idea: custom base_url + key + model.

4. Pass tools through the framework, not ad hoc HTTP

Let the framework bind tools; keep OpenRouter as transport.

5. Verify tool-capable models per framework path

A model that streams fine may still fail structured tool calls inside a given integration version.

6. Log framework run ids with model slug

When quality drops, you need both the graph node and the resolved OpenRouter model.

Working Example

Shared factory (conceptual; pin package APIs at build):

import os
from functools import lru_cache
 
OPENROUTER_BASE = os.getenv(
    "OPENROUTER_BASE_URL", "https://openrouter.ai/api/v1"
)
API_KEY = os.environ["OPENROUTER_API_KEY"]
REGISTRY = {
    "default": os.getenv("AGENT_MODEL_DEFAULT", "openai/gpt-4o-mini"),
    "planner": os.getenv("AGENT_MODEL_PLANNER", "anthropic/claude-sonnet-4"),
}
 
HEADERS = {
    "HTTP-Referer": os.getenv("APP_URL", "https://example.com"),
    "X-Title": os.getenv("APP_NAME", "agent-prod"),
}
 
@lru_cache
def model_id(role: str = "default") -> str:
    return REGISTRY[role]

LangChain / LangGraph (langchain-openai; verify imports for your version):

from langchain_openai import ChatOpenAI
 
def lc_model(role: str = "default") -> ChatOpenAI:
    return ChatOpenAI(
        model=model_id(role),
        api_key=API_KEY,
        base_url=OPENROUTER_BASE,
        default_headers=HEADERS,
        temperature=0,
    )

LlamaIndex (OpenAI-compatible pattern; verify class names at build):

from llama_index.llms.openai import OpenAI as LlamaOpenAI
 
def li_model(role: str = "default") -> LlamaOpenAI:
    return LlamaOpenAI(
        model=model_id(role),
        api_key=API_KEY,
        api_base=OPENROUTER_BASE,
        default_headers=HEADERS,
    )

CrewAI (often via LiteLLM-style model strings; verify current CrewAI docs):

from crewai import Agent, LLM
 
def crew_llm(role: str = "default") -> LLM:
    # Many setups use openrouter/<slug> with OPENROUTER_API_KEY set
    return LLM(
        model=f"openrouter/{model_id(role)}",
        api_key=API_KEY,
        base_url=OPENROUTER_BASE,
    )
 
researcher = Agent(
    role="Researcher",
    goal="Find grounded facts",
    backstory="Careful analyst",
    llm=crew_llm("default"),
    verbose=False,
)

If your CrewAI version expects a different LLM shape, keep the same principle: one factory, OpenRouter base, registry slug.

Deep Dive

Why frameworks + OpenRouter work well

Frameworks own orchestration: graphs, crews, agents, memory hooks, tool binders.

OpenRouter owns model access: multi-provider inventory, unified auth, optional provider routing and fallbacks.

That split matches Architecting an Agent Around a Model-Agnostic Gateway.

Per-framework notes

Multiple models in one graph or crew

Use role-based registry entries:

• planner / manager → stronger slug
• worker / extractor → cheaper slug
• fallback → secondary provider slug

Wire roles in config, not by forking agent code.

Streaming and async

All three ecosystems support streaming to varying degrees.

Test streaming + tool calls together; some paths buffer tool calls until the stream segment completes.

Dependencies and versions

Framework packages move quickly.

Pin langchain-*, llama-index-*, and crewai versions in the app lockfile, and re-verify OpenRouter base URL parameters after upgrades (verify at build).

Testing strategy

1. Unit-test graphs with a fake chat model.
2. Smoke-test OpenRouter with one cheap model and one tool.
3. Eval suite on the exact slugs registered for prod.

Gotchas

1. Using OPENAI_API_KEY only - Some libs read OpenAI env vars by default. Set the key you pass explicitly to OpenRouter's key.
2. Wrong base URL path - Missing /api/v1 or trailing-slash mismatches cause confusing 404s.
3. Model slug without provider prefix - OpenRouter typically expects vendor/model form for many models.
4. Per-agent client sprawl - Ten slightly different temperatures and keys make incidents undebuggable.
5. Assuming CrewAI "LLM" equals LangChain model object - Do not mix objects across frameworks; only share registry and env.
6. Silent tool disable - Framework may no-op tools if the model integration lacks tool support. Fail tests when tools are configured but never invoked.
7. Double markup of cost - Framework retries plus OpenRouter fallbacks can multiply spend. Cap both layers.

Alternatives

FAQs

Related

• Production OpenRouter Basics - client without a framework
• Architecting an Agent Around a Model-Agnostic Gateway - design pattern
• Tool Calling Through OpenRouter Across Different Providers - tools
• Case Study: Migrating a Single-Provider Agent to OpenRouter - migration
• OpenRouter in Production Best Practices - checklist

Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana @solana/web3.js + @solana/spl-token.