Combining OpenRouter with LangChain, CrewAI, and LlamaIndex

Most agent frameworks already speak OpenAI-compatible chat APIs.

OpenRouter exposes that shape at https://openrouter.ai/api/v1, so you can keep LangChain, CrewAI, or LlamaIndex orchestration and swap only the model backend.

This page gives working wiring patterns and the production rules that stop "works in the tutorial" setups from leaking keys or hardcoding SKUs.

Summary

Construct each framework's chat model once through a shared factory: OpenRouter base URL, API key, model slug, and optional headers.

Do not configure OpenRouter differently in every agent class.

Recipe

1. Centralize env and registry

export OPENROUTER_API_KEY=sk-or-...
export OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
export AGENT_MODEL_DEFAULT=openai/gpt-4o-mini

Map logical roles (default, planner, researcher) to slugs in one module.

2. Build models only in a factory

Every framework entrypoint calls get_chat_model(role) rather than constructing clients inline.

3. Prefer OpenAI-compatible adapters

LangChain ChatOpenAI, LlamaIndex OpenAI-compatible LLMs, and CrewAI/LiteLLM OpenRouter routes all work from the same idea: custom base_url + key + model.

4. Pass tools through the framework, not ad hoc HTTP

Let the framework bind tools; keep OpenRouter as transport.

5. Verify tool-capable models per framework path

A model that streams fine may still fail structured tool calls inside a given integration version.

6. Log framework run ids with model slug

When quality drops, you need both the graph node and the resolved OpenRouter model.

Working Example

Shared factory (conceptual; pin package APIs at build):

import os
from functools import lru_cache
 
OPENROUTER_BASE = os.getenv(
    "OPENROUTER_BASE_URL", "https://openrouter.ai/api/v1"
)
API_KEY = os.environ["OPENROUTER_API_KEY"]
REGISTRY = {
    "default": os.getenv("AGENT_MODEL_DEFAULT", "openai/gpt-4o-mini"),
    "planner": os.getenv("AGENT_MODEL_PLANNER", "anthropic/claude-sonnet-4"),
}
 
HEADERS = {
    "HTTP-Referer": os.getenv("APP_URL", "https://example.com"),
    "X-Title": os.getenv("APP_NAME", "agent-prod"),
}
 
@lru_cache
def model_id(role: str = "default") -> str:
    return REGISTRY[role]

LangChain / LangGraph (langchain-openai; verify imports for your version):

from langchain_openai import ChatOpenAI
 
def lc_model(role: str = "default") -> ChatOpenAI:
    return ChatOpenAI(
        model=model_id(role),
        api_key=API_KEY,
        base_url=OPENROUTER_BASE,
        default_headers=HEADERS,
        temperature=0,
    )

LlamaIndex (OpenAI-compatible pattern; verify class names at build):

from llama_index.llms.openai import OpenAI as LlamaOpenAI
 
def li_model(role: str = "default") -> LlamaOpenAI:
    return LlamaOpenAI(
        model=model_id(role),
        api_key=API_KEY,
        api_base=OPENROUTER_BASE,
        default_headers=HEADERS,
    )

CrewAI (often via LiteLLM-style model strings; verify current CrewAI docs):

from crewai import Agent, LLM
 
def crew_llm(role: str = "default") -> LLM:
    # Many setups use openrouter/<slug> with OPENROUTER_API_KEY set
    return LLM(
        model=f"openrouter/{model_id(role)}",
        api_key=API_KEY,
        base_url=OPENROUTER_BASE,
    )
 
researcher = Agent(
    role="Researcher",
    goal="Find grounded facts",
    backstory="Careful analyst",
    llm=crew_llm("default"),
    verbose=False,
)

If your CrewAI version expects a different LLM shape, keep the same principle: one factory, OpenRouter base, registry slug.

Deep Dive

Why frameworks + OpenRouter work well

Frameworks own orchestration: graphs, crews, agents, memory hooks, tool binders.

OpenRouter owns model access: multi-provider inventory, unified auth, optional provider routing and fallbacks.

That split matches Architecting an Agent Around a Model-Agnostic Gateway.

Per-framework notes

Framework	Typical integration	Watch-outs
LangChain / LangGraph	`ChatOpenAI` + `base_url`	Bind tools on the model instance used by the node; do not recreate clients per token
LlamaIndex	OpenAI-compatible LLM classes	Align chat vs complete APIs; agent runners need tool-calling models
CrewAI	`LLM` / LiteLLM `openrouter/...`	Hierarchical crews multiply calls - set budgets; pin model per agent role

Multiple models in one graph or crew

Use role-based registry entries:

planner / manager → stronger slug
worker / extractor → cheaper slug
fallback → secondary provider slug

Wire roles in config, not by forking agent code.

Streaming and async

All three ecosystems support streaming to varying degrees.

Test streaming + tool calls together; some paths buffer tool calls until the stream segment completes.

Dependencies and versions

Framework packages move quickly.

Pin langchain-*, llama-index-*, and crewai versions in the app lockfile, and re-verify OpenRouter base URL parameters after upgrades (verify at build).

Testing strategy

Unit-test graphs with a fake chat model.
Smoke-test OpenRouter with one cheap model and one tool.
Eval suite on the exact slugs registered for prod.

Gotchas

Using OPENAI_API_KEY only - Some libs read OpenAI env vars by default. Set the key you pass explicitly to OpenRouter's key.
Wrong base URL path - Missing /api/v1 or trailing-slash mismatches cause confusing 404s.
Model slug without provider prefix - OpenRouter typically expects vendor/model form for many models.
Per-agent client sprawl - Ten slightly different temperatures and keys make incidents undebuggable.
Assuming CrewAI "LLM" equals LangChain model object - Do not mix objects across frameworks; only share registry and env.
Silent tool disable - Framework may no-op tools if the model integration lacks tool support. Fail tests when tools are configured but never invoked.
Double markup of cost - Framework retries plus OpenRouter fallbacks can multiply spend. Cap both layers.

Alternatives

Approach	Strength	Weakness	Best fit
Framework + OpenRouter	Fast multi-model orchestration	Two version surfaces	Most product teams
Framework + direct OpenAI/Anthropic	Native SDK features	Harder multi-provider	Single-provider mandate
Framework-free + OpenRouter client	Maximum control	You build memory/tools	Small custom runtimes
LiteLLM proxy in front of everything	Central policy	Extra network hop	Platform teams

FAQs

Can one OpenRouter key serve LangChain and CrewAI in the same app?

Yes.

Share the key and factory; isolate only if you need separate spend caps or environments.

Do I need LangChain to use OpenRouter?

No.

OpenRouter works with any OpenAI-compatible client. Frameworks are optional orchestration.

How do I set provider preferences (order, fallbacks) from LangChain?

Pass extra body fields supported by your chat model wrapper (for example extra_body on OpenAI-compatible clients). Verify field names against OpenRouter docs at build.

Should every CrewAI agent use a different model?

Only when roles differ in difficulty or risk. Start with one default; split when evals or cost data justify it.

Does LlamaIndex RAG require OpenRouter embeddings too?

No.

You can use OpenRouter (or another provider) for the chat LLM and a different embeddings provider. Keep configs explicit.

How do I migrate from OpenAI-only LangChain code?

Change base_url, key, and model strings to OpenRouter slugs; run tool and RAG evals; then add fallbacks.

Are free OpenRouter models fine for framework demos?

For demos, often yes. For tool-using production agents, only if they pass your tool evals and rate limits.

Where should I put HTTP-Referer and X-Title?

In the shared factory default headers so every framework path attributes traffic consistently.

Can I A/B test models inside LangGraph?

Yes.

Resolve model_id from a flag or experiment assignment before building or selecting the node model. See A/B Testing Models in Production via OpenRouter.

What breaks first when upgrading frameworks?

Import paths, tool binding helpers, and message content block shapes. Keep a thin adapter and a smoke test.

Production OpenRouter Basics - client without a framework
Architecting an Agent Around a Model-Agnostic Gateway - design pattern
Tool Calling Through OpenRouter Across Different Providers - tools
Case Study: Migrating a Single-Provider Agent to OpenRouter - migration
OpenRouter in Production Best Practices - checklist

Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana @solana/web3.js + @solana/spl-token.

Combining OpenRouter with LangChain, CrewAI, and LlamaIndex

Most agent frameworks already speak OpenAI-compatible chat APIs.

OpenRouter exposes that shape at https://openrouter.ai/api/v1, so you can keep LangChain, CrewAI, or LlamaIndex orchestration and swap only the model backend.

This page gives working wiring patterns and the production rules that stop "works in the tutorial" setups from leaking keys or hardcoding SKUs.

Summary

Construct each framework's chat model once through a shared factory: OpenRouter base URL, API key, model slug, and optional headers.

Do not configure OpenRouter differently in every agent class.

Recipe

1. Centralize env and registry

export OPENROUTER_API_KEY=sk-or-...
export OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
export AGENT_MODEL_DEFAULT=openai/gpt-4o-mini

Map logical roles (default, planner, researcher) to slugs in one module.

2. Build models only in a factory

Every framework entrypoint calls get_chat_model(role) rather than constructing clients inline.

3. Prefer OpenAI-compatible adapters

LangChain ChatOpenAI, LlamaIndex OpenAI-compatible LLMs, and CrewAI/LiteLLM OpenRouter routes all work from the same idea: custom base_url + key + model.

4. Pass tools through the framework, not ad hoc HTTP

Let the framework bind tools; keep OpenRouter as transport.

5. Verify tool-capable models per framework path

A model that streams fine may still fail structured tool calls inside a given integration version.

6. Log framework run ids with model slug

When quality drops, you need both the graph node and the resolved OpenRouter model.

Working Example

Shared factory (conceptual; pin package APIs at build):

import os
from functools import lru_cache
 
OPENROUTER_BASE = os.getenv(
    "OPENROUTER_BASE_URL", "https://openrouter.ai/api/v1"
)
API_KEY = os.environ["OPENROUTER_API_KEY"]
REGISTRY = {
    "default": os.getenv("AGENT_MODEL_DEFAULT", "openai/gpt-4o-mini"),
    "planner": os.getenv("AGENT_MODEL_PLANNER", "anthropic/claude-sonnet-4"),
}
 
HEADERS = {
    "HTTP-Referer": os.getenv("APP_URL", "https://example.com"),
    "X-Title": os.getenv("APP_NAME", "agent-prod"),
}
 
@lru_cache
def model_id(role: str = "default") -> str:
    return REGISTRY[role]

LangChain / LangGraph (langchain-openai; verify imports for your version):

from langchain_openai import ChatOpenAI
 
def lc_model(role: str = "default") -> ChatOpenAI:
    return ChatOpenAI(
        model=model_id(role),
        api_key=API_KEY,
        base_url=OPENROUTER_BASE,
        default_headers=HEADERS,
        temperature=0,
    )

LlamaIndex (OpenAI-compatible pattern; verify class names at build):

from llama_index.llms.openai import OpenAI as LlamaOpenAI
 
def li_model(role: str = "default") -> LlamaOpenAI:
    return LlamaOpenAI(
        model=model_id(role),
        api_key=API_KEY,
        api_base=OPENROUTER_BASE,
        default_headers=HEADERS,
    )

CrewAI (often via LiteLLM-style model strings; verify current CrewAI docs):

from crewai import Agent, LLM
 
def crew_llm(role: str = "default") -> LLM:
    # Many setups use openrouter/<slug> with OPENROUTER_API_KEY set
    return LLM(
        model=f"openrouter/{model_id(role)}",
        api_key=API_KEY,
        base_url=OPENROUTER_BASE,
    )
 
researcher = Agent(
    role="Researcher",
    goal="Find grounded facts",
    backstory="Careful analyst",
    llm=crew_llm("default"),
    verbose=False,
)

If your CrewAI version expects a different LLM shape, keep the same principle: one factory, OpenRouter base, registry slug.

Deep Dive

Why frameworks + OpenRouter work well

Frameworks own orchestration: graphs, crews, agents, memory hooks, tool binders.

OpenRouter owns model access: multi-provider inventory, unified auth, optional provider routing and fallbacks.

That split matches Architecting an Agent Around a Model-Agnostic Gateway.

Per-framework notes

Framework	Typical integration	Watch-outs
LangChain / LangGraph	`ChatOpenAI` + `base_url`	Bind tools on the model instance used by the node; do not recreate clients per token
LlamaIndex	OpenAI-compatible LLM classes	Align chat vs complete APIs; agent runners need tool-calling models
CrewAI	`LLM` / LiteLLM `openrouter/...`	Hierarchical crews multiply calls - set budgets; pin model per agent role

Multiple models in one graph or crew

Use role-based registry entries:

planner / manager → stronger slug
worker / extractor → cheaper slug
fallback → secondary provider slug

Wire roles in config, not by forking agent code.

Streaming and async

All three ecosystems support streaming to varying degrees.

Test streaming + tool calls together; some paths buffer tool calls until the stream segment completes.

Dependencies and versions

Framework packages move quickly.

Pin langchain-*, llama-index-*, and crewai versions in the app lockfile, and re-verify OpenRouter base URL parameters after upgrades (verify at build).

Testing strategy

Unit-test graphs with a fake chat model.
Smoke-test OpenRouter with one cheap model and one tool.
Eval suite on the exact slugs registered for prod.

Gotchas

Using OPENAI_API_KEY only - Some libs read OpenAI env vars by default. Set the key you pass explicitly to OpenRouter's key.
Wrong base URL path - Missing /api/v1 or trailing-slash mismatches cause confusing 404s.
Model slug without provider prefix - OpenRouter typically expects vendor/model form for many models.
Per-agent client sprawl - Ten slightly different temperatures and keys make incidents undebuggable.
Assuming CrewAI "LLM" equals LangChain model object - Do not mix objects across frameworks; only share registry and env.
Silent tool disable - Framework may no-op tools if the model integration lacks tool support. Fail tests when tools are configured but never invoked.
Double markup of cost - Framework retries plus OpenRouter fallbacks can multiply spend. Cap both layers.

Alternatives

Approach	Strength	Weakness	Best fit
Framework + OpenRouter	Fast multi-model orchestration	Two version surfaces	Most product teams
Framework + direct OpenAI/Anthropic	Native SDK features	Harder multi-provider	Single-provider mandate
Framework-free + OpenRouter client	Maximum control	You build memory/tools	Small custom runtimes
LiteLLM proxy in front of everything	Central policy	Extra network hop	Platform teams

FAQs

Can one OpenRouter key serve LangChain and CrewAI in the same app?

Yes.

Share the key and factory; isolate only if you need separate spend caps or environments.

Do I need LangChain to use OpenRouter?

No.

OpenRouter works with any OpenAI-compatible client. Frameworks are optional orchestration.

How do I set provider preferences (order, fallbacks) from LangChain?

Pass extra body fields supported by your chat model wrapper (for example extra_body on OpenAI-compatible clients). Verify field names against OpenRouter docs at build.

Should every CrewAI agent use a different model?

Only when roles differ in difficulty or risk. Start with one default; split when evals or cost data justify it.

Does LlamaIndex RAG require OpenRouter embeddings too?

No.

You can use OpenRouter (or another provider) for the chat LLM and a different embeddings provider. Keep configs explicit.

How do I migrate from OpenAI-only LangChain code?

Change base_url, key, and model strings to OpenRouter slugs; run tool and RAG evals; then add fallbacks.

Are free OpenRouter models fine for framework demos?

For demos, often yes. For tool-using production agents, only if they pass your tool evals and rate limits.

Where should I put HTTP-Referer and X-Title?

In the shared factory default headers so every framework path attributes traffic consistently.

Can I A/B test models inside LangGraph?

Yes.

Resolve model_id from a flag or experiment assignment before building or selecting the node model. See A/B Testing Models in Production via OpenRouter.

What breaks first when upgrading frameworks?

Import paths, tool binding helpers, and message content block shapes. Keep a thin adapter and a smoke test.

Production OpenRouter Basics - client without a framework
Architecting an Agent Around a Model-Agnostic Gateway - design pattern
Tool Calling Through OpenRouter Across Different Providers - tools
Case Study: Migrating a Single-Provider Agent to OpenRouter - migration
OpenRouter in Production Best Practices - checklist

Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana @solana/web3.js + @solana/spl-token.

Combining OpenRouter with LangChain, CrewAI, and LlamaIndex

Search Documentation

Combining OpenRouter with LangChain, CrewAI, and LlamaIndex