Why Some Teams Build a Custom Agent Runtime Instead of Adopting a Framework

A custom agent runtime is your own control plane: you own the model call, tool dispatch, stop rules, and run metadata without LangGraph, CrewAI, Microsoft Agent Framework, or a provider Agents SDK sitting in the middle. Some teams pick that path deliberately. This page explains when that decision is rational and when it is not-invented-here dressed up as architecture.

Summary

Hand-rolled runtimes win when orchestration needs are small, dependency and compliance constraints are hard, or framework abstractions fight your product shape.
Insight: Frameworks speed demos and multi-pattern orchestration; they also add upgrade risk, opaque magic, and lock-in around tools and state.
Key Concepts: agent runtime, thin loop, control plane, dependency budget, undifferentiated heavy lifting, escape hatch.
When to Use: Writing an ADR, reviewing a POC that "just used LangGraph," or deciding whether a provider Agents SDK is already enough without a third-party graph library.
Limitations/Trade-offs: Custom still uses HTTP clients, queues, and OpenTelemetry. Framework-free means you own the loop, not that you invent TLS or JSON Schema.
Related Topics: framework-free trade-offs, ReAct loops as APIs, eval harnesses, when to abandon the custom runtime.

Foundations

An agent runtime is the host that repeatedly: builds context, calls a model, executes tools, appends observations, enforces budgets, and emits a final result or stop reason.

A framework-based runtime borrows that host from a library (LangGraph graphs, CrewAI crews, OpenAI Agents Runner, Pydantic AI agents, and peers).

A custom runtime keeps those semantics in your service code, usually as a bounded loop behind FastAPI, Express, a worker, or a CLI.

Framework path:
  App -> Framework control plane -> Model + tools -> Framework state helpers
 
Custom path:
  App owns loop -> Model client -> Tool plugins -> Your store + traces

Neither path is "no dependencies." The question is who owns orchestration truth: stop enums, tool allowlists, retries, and what a "run" means in your product.

Teams that build custom usually share at least one pressure:

Dependency freezes - regulated or air-gapped environments reject fast-moving agent frameworks.
Simple loop shape - tools + budgets, not durable multi-day graphs with joins.
Strict observability contracts - every field must land in their warehouse schema, not a vendor UI alone.
Product is the runtime - they sell agent hosting and cannot ship a third-party control plane as the core.
Framework mismatch - they tried a framework and spent more time fighting its graph model than shipping tools.

Analogy: buying a workflow engine versus writing a state machine for one process. One process with clear stop rules can be a 150-line loop. Twenty processes with pause/resume and visual ops usually want a platform.

Mechanics & Interactions

What custom actually buys

Benefit	What it looks like in practice
Full stop control	`max_turns`, wall clock, cost caps, and cancel flags in your code path
Transparent tool path	Dispatcher you can unit test without framework mocks
Stable domain contracts	Tools stay pure functions; the loop is a thin adapter
Predictable upgrades	Model SDK upgrades only; no graph DSL major every quarter
Compliance narrative	You can point auditors at the exact file that opens circuits and denies tools

What you still pay for

Cost	Symptom if ignored
You own retries and backoff	Flaky tools thrash the model into spend
You own parallel tool semantics	Partial failures become inconsistent transcripts
You own multi-agent later	Ad-hoc handoffs become an undocumented second framework
You own HITL resume	"Just add a queue" becomes a half-built workflow engine
You own docs and hiring	New engineers learn your loop, not a public skill market

Custom vs native provider SDKs vs third-party frameworks

Approach	Owns the loop	Best when
Custom thin runtime	You	Hard constraints + simple tool loop
Native Agents SDK (OpenAI, Google ADK, Anthropic Messages loop)	Vendor/you mix	Single-provider product, handoffs/MCP wanted fast
Third-party framework	Library	Graphs, crews, enterprise connectors, multi-surface ops

Custom is not automatically more portable than a framework. Portability comes from pure tools, versioned schemas, and prompts as data. A custom loop that embeds business rules only in prompt strings is still a trap.

Signals that custom is rational

The product needs a single bounded ReAct-style loop for months, not branching workflows.
Security review will not approve large orchestration packages without months of legal work.
The team already operates solid HTTP services and prefers boring code over framework lore.
You can staff ownership: one named engineer/team owns stop rules, evals, and on-call for the loop.
A one-week spike shows the framework adding escape hatches faster than features.

Signals that custom is cosplay

You are already implementing generic checkpoint packages, join semantics, and a plugin graph DSL.
Product needs multi-day human approval with resume across deploys.
Five agents and fan-out/fan-in appear on the roadmap this quarter.
Nobody owns runtime reliability; "the loop" lives in a notebook and a Slack thread.
The real motivation is dislike of docs, not a measured dependency or control problem.

Interaction with the rest of the stack

Custom runtimes still plug into:

Model gateways (direct provider or OpenRouter-style routing)
Tool plugins (local functions, MCP clients, internal APIs)
Eval harnesses (gold sets, LLM-as-judge, CI gates)
Observability (structured logs, OpenTelemetry spans, cost metrics)
Product APIs (FastAPI/Express endpoints that start runs and stream events)

The runtime is the seam between product HTTP and model I/O. Keep that seam thin so domain services do not import "agent" types.

Advanced Considerations & Applications

Layering that keeps an exit hatch

domain/tools/*          # pure, testable side effects
domain/schemas/*        # JSON Schema / Pydantic contracts
runtime/loop.py         # model call + dispatch + budgets
runtime/plugins/*       # tool registration adapters
adapters/http.py        # FastAPI / Express surface

If you later adopt LangGraph or a native Agents SDK, you rewrite runtime/* and keep domain/*.

Trade-off table

Approach	Best fit	Main cost
Custom thin loop	Simple agents, compliance niches, platform builders	You own reliability forever
Native provider SDK	Single-provider products needing handoffs/MCP fast	Provider coupling
Full third-party framework	Complex orchestration and shared multi-product ops	Dependency weight and dual learning curves
Custom + borrowed pieces	Thin loop + OTel + your DB checkpoints	Discipline not to reinvent graphs

When custom becomes a framework anyway

Watch for these code smells:

A private WorkflowEngine with more docs than your product
YAML DSLs for agent graphs "just for us"
Reimplemented memory, RAG, and multi-agent routers without product need
Incompatible loop copies in three services

At that point you did not avoid a framework. You forked one without community support. See When a Custom Runtime Stops Being Worth Maintaining.

Organizational fit

Team shape	Custom tends to…
Small backend-strong team, one agent product	Work well
Platform team shipping agent host as product	Work well if staffed
Large org with many agent apps	Prefer shared framework or internal platform once
Frontend-heavy team needing graphs	Struggle without framework DX

Skill match matters as much as architecture. See Matching Team Skill Set to Framework Choice.

Measuring whether custom paid off

Track:

Days from spike to first production tool path
Share of agent code in domain/ vs runtime/
Incident count from missing stop conditions
Cost of adopting one new model provider (should be small)
Hours spent reimplementing features frameworks already ship

If provider swaps are cheap and stop-related incidents stay near zero, custom is earning rent. If you spend every sprint on generic orchestration, migrate.

Common Misconceptions

"Custom means zero dependencies." You still depend on model SDKs, HTTP stacks, and observability libraries.
"Frameworks are always slower." For complex graphs, a mature framework is often faster to correctness than a buggy custom state machine.
"Native Agents SDKs are not frameworks." They are provider-aligned frameworks; custom is a third option beside them and multi-vendor libs.
"We can add stop conditions later." Unbounded loops are a production incident generator; budgets belong in v1.
"Custom is more secure by default." Security comes from tool least privilege, sandboxes, and authZ - not from avoiding pip packages.
"If we write it, we understand it." Only if it is small, tested, and documented. A 3k-line private orchestrator is not automatic clarity.

FAQs

Is a custom runtime the same as calling Chat Completions in a while-loop?

Often yes at the core. Production custom runtimes add budgets, structured stop reasons, tool plugins, auth context, tracing, and an HTTP or worker API around that loop.

How is this different from OpenAI Agents SDK or Anthropic tool loops?

Provider kits give official agent objects, handoffs, guardrails, or Messages tool primitives. Custom means you own those pieces (or deliberately omit them) without adopting that kit as your control plane.

When is LangGraph still the better default?

When durable resume, complex branching, joins, or multi-path HITL are first-class product needs. Reimplementing those well is expensive.

Can we use OpenRouter inside a custom runtime?

Yes. Treat OpenRouter (or any gateway) as the model client behind your loop. Routing is orthogonal to who owns the agent loop.

What should never live only in the runtime module?

Business invariants, authorization, durable customer records, and tool implementations other services must call. Keep those in domain modules with independent tests.

How big should a "thin" custom loop stay?

If the loop file grows past a few hundred lines of generic orchestration (graphs, joins, DSL parsers), you are inventing a framework. Extract only product-specific policy or switch stacks.

Do custom runtimes support streaming UIs?

Yes. Stream token or event chunks from the host; keep privileged tools on the server. UI SDKs (for example Vercel AI SDK) can present streams without owning the runtime.

Is MCP compatible with custom runtimes?

Yes. MCP is a tool-connection protocol. Your dispatcher can list MCP tools, translate schemas, and call MCP clients like any other plugin source.

What belongs in the ADR when we choose custom?

Drivers (compliance, simplicity, platform product), alternatives considered, stop/budget requirements, ownership, eval plan, and explicit exit triggers that force migration.

How do we prevent each team from forking a private loop?

Publish an internal thin runtime package with plugin interfaces, or standardize on one framework. Custom without a shared platform multiplies operational cost.

Are evals optional if the loop is small?

No. Small loops still change prompts, models, and tools. Gold-set evals and CI gates are how you know the custom path still works after edits.

When should we abandon custom for a maintained framework?

When roadmap needs outgrow the loop, staffing cannot own reliability, or you are reimplementing generic orchestration weekly. Use the decision cheatsheet in this section.

Custom Agent Runtimes Basics - first hand-rolled loop and dispatcher
Designing a ReAct Loop from Scratch in FastAPI or Express - API-shaped recipe
When a Custom Runtime Stops Being Worth Maintaining - exit checklist
Custom Agent Runtimes Best Practices - maintainability practices
Framework-Free vs Framework-Based: When to Roll Your Own - complementary trade-off frame
What Native Provider Agent Primitives Give You Over a Framework - native SDK alternative

Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana @solana/web3.js + @solana/spl-token.

Summary

Hand-rolled runtimes win when orchestration needs are small, dependency and compliance constraints are hard, or framework abstractions fight your product shape.
Insight: Frameworks speed demos and multi-pattern orchestration; they also add upgrade risk, opaque magic, and lock-in around tools and state.
Key Concepts: agent runtime, thin loop, control plane, dependency budget, undifferentiated heavy lifting, escape hatch.
When to Use: Writing an ADR, reviewing a POC that "just used LangGraph," or deciding whether a provider Agents SDK is already enough without a third-party graph library.
Limitations/Trade-offs: Custom still uses HTTP clients, queues, and OpenTelemetry. Framework-free means you own the loop, not that you invent TLS or JSON Schema.
Related Topics: framework-free trade-offs, ReAct loops as APIs, eval harnesses, when to abandon the custom runtime.

Foundations

An agent runtime is the host that repeatedly: builds context, calls a model, executes tools, appends observations, enforces budgets, and emits a final result or stop reason.

A framework-based runtime borrows that host from a library (LangGraph graphs, CrewAI crews, OpenAI Agents Runner, Pydantic AI agents, and peers).

A custom runtime keeps those semantics in your service code, usually as a bounded loop behind FastAPI, Express, a worker, or a CLI.

Framework path:
  App -> Framework control plane -> Model + tools -> Framework state helpers
 
Custom path:
  App owns loop -> Model client -> Tool plugins -> Your store + traces

Neither path is "no dependencies." The question is who owns orchestration truth: stop enums, tool allowlists, retries, and what a "run" means in your product.

Teams that build custom usually share at least one pressure:

Dependency freezes - regulated or air-gapped environments reject fast-moving agent frameworks.
Simple loop shape - tools + budgets, not durable multi-day graphs with joins.
Strict observability contracts - every field must land in their warehouse schema, not a vendor UI alone.
Product is the runtime - they sell agent hosting and cannot ship a third-party control plane as the core.
Framework mismatch - they tried a framework and spent more time fighting its graph model than shipping tools.

Mechanics & Interactions

What custom actually buys

Benefit	What it looks like in practice
Full stop control	`max_turns`, wall clock, cost caps, and cancel flags in your code path
Transparent tool path	Dispatcher you can unit test without framework mocks
Stable domain contracts	Tools stay pure functions; the loop is a thin adapter
Predictable upgrades	Model SDK upgrades only; no graph DSL major every quarter
Compliance narrative	You can point auditors at the exact file that opens circuits and denies tools

What you still pay for

Cost	Symptom if ignored
You own retries and backoff	Flaky tools thrash the model into spend
You own parallel tool semantics	Partial failures become inconsistent transcripts
You own multi-agent later	Ad-hoc handoffs become an undocumented second framework
You own HITL resume	"Just add a queue" becomes a half-built workflow engine
You own docs and hiring	New engineers learn your loop, not a public skill market

Custom vs native provider SDKs vs third-party frameworks

Approach	Owns the loop	Best when
Custom thin runtime	You	Hard constraints + simple tool loop
Native Agents SDK (OpenAI, Google ADK, Anthropic Messages loop)	Vendor/you mix	Single-provider product, handoffs/MCP wanted fast
Third-party framework	Library	Graphs, crews, enterprise connectors, multi-surface ops

Signals that custom is rational

The product needs a single bounded ReAct-style loop for months, not branching workflows.
Security review will not approve large orchestration packages without months of legal work.
The team already operates solid HTTP services and prefers boring code over framework lore.
You can staff ownership: one named engineer/team owns stop rules, evals, and on-call for the loop.
A one-week spike shows the framework adding escape hatches faster than features.

Signals that custom is cosplay

You are already implementing generic checkpoint packages, join semantics, and a plugin graph DSL.
Product needs multi-day human approval with resume across deploys.
Five agents and fan-out/fan-in appear on the roadmap this quarter.
Nobody owns runtime reliability; "the loop" lives in a notebook and a Slack thread.
The real motivation is dislike of docs, not a measured dependency or control problem.

Interaction with the rest of the stack

Custom runtimes still plug into:

Model gateways (direct provider or OpenRouter-style routing)
Tool plugins (local functions, MCP clients, internal APIs)
Eval harnesses (gold sets, LLM-as-judge, CI gates)
Observability (structured logs, OpenTelemetry spans, cost metrics)
Product APIs (FastAPI/Express endpoints that start runs and stream events)

The runtime is the seam between product HTTP and model I/O. Keep that seam thin so domain services do not import "agent" types.

Advanced Considerations & Applications

Layering that keeps an exit hatch

domain/tools/*          # pure, testable side effects
domain/schemas/*        # JSON Schema / Pydantic contracts
runtime/loop.py         # model call + dispatch + budgets
runtime/plugins/*       # tool registration adapters
adapters/http.py        # FastAPI / Express surface

If you later adopt LangGraph or a native Agents SDK, you rewrite runtime/* and keep domain/*.

Trade-off table

Approach	Best fit	Main cost
Custom thin loop	Simple agents, compliance niches, platform builders	You own reliability forever
Native provider SDK	Single-provider products needing handoffs/MCP fast	Provider coupling
Full third-party framework	Complex orchestration and shared multi-product ops	Dependency weight and dual learning curves
Custom + borrowed pieces	Thin loop + OTel + your DB checkpoints	Discipline not to reinvent graphs

When custom becomes a framework anyway

Watch for these code smells:

A private WorkflowEngine with more docs than your product
YAML DSLs for agent graphs "just for us"
Reimplemented memory, RAG, and multi-agent routers without product need
Incompatible loop copies in three services

At that point you did not avoid a framework. You forked one without community support. See When a Custom Runtime Stops Being Worth Maintaining.

Organizational fit

Team shape	Custom tends to…
Small backend-strong team, one agent product	Work well
Platform team shipping agent host as product	Work well if staffed
Large org with many agent apps	Prefer shared framework or internal platform once
Frontend-heavy team needing graphs	Struggle without framework DX

Skill match matters as much as architecture. See Matching Team Skill Set to Framework Choice.

Measuring whether custom paid off

Track:

Days from spike to first production tool path
Share of agent code in domain/ vs runtime/
Incident count from missing stop conditions
Cost of adopting one new model provider (should be small)
Hours spent reimplementing features frameworks already ship

If provider swaps are cheap and stop-related incidents stay near zero, custom is earning rent. If you spend every sprint on generic orchestration, migrate.

Common Misconceptions

"Custom means zero dependencies." You still depend on model SDKs, HTTP stacks, and observability libraries.
"Frameworks are always slower." For complex graphs, a mature framework is often faster to correctness than a buggy custom state machine.
"Native Agents SDKs are not frameworks." They are provider-aligned frameworks; custom is a third option beside them and multi-vendor libs.
"We can add stop conditions later." Unbounded loops are a production incident generator; budgets belong in v1.
"Custom is more secure by default." Security comes from tool least privilege, sandboxes, and authZ - not from avoiding pip packages.
"If we write it, we understand it." Only if it is small, tested, and documented. A 3k-line private orchestrator is not automatic clarity.

FAQs

Is a custom runtime the same as calling Chat Completions in a while-loop?

Often yes at the core. Production custom runtimes add budgets, structured stop reasons, tool plugins, auth context, tracing, and an HTTP or worker API around that loop.

How is this different from OpenAI Agents SDK or Anthropic tool loops?

When is LangGraph still the better default?

When durable resume, complex branching, joins, or multi-path HITL are first-class product needs. Reimplementing those well is expensive.

Can we use OpenRouter inside a custom runtime?

Yes. Treat OpenRouter (or any gateway) as the model client behind your loop. Routing is orthogonal to who owns the agent loop.

What should never live only in the runtime module?

Business invariants, authorization, durable customer records, and tool implementations other services must call. Keep those in domain modules with independent tests.

How big should a "thin" custom loop stay?

If the loop file grows past a few hundred lines of generic orchestration (graphs, joins, DSL parsers), you are inventing a framework. Extract only product-specific policy or switch stacks.

Do custom runtimes support streaming UIs?

Yes. Stream token or event chunks from the host; keep privileged tools on the server. UI SDKs (for example Vercel AI SDK) can present streams without owning the runtime.

Is MCP compatible with custom runtimes?

Yes. MCP is a tool-connection protocol. Your dispatcher can list MCP tools, translate schemas, and call MCP clients like any other plugin source.

What belongs in the ADR when we choose custom?

Drivers (compliance, simplicity, platform product), alternatives considered, stop/budget requirements, ownership, eval plan, and explicit exit triggers that force migration.

How do we prevent each team from forking a private loop?

Publish an internal thin runtime package with plugin interfaces, or standardize on one framework. Custom without a shared platform multiplies operational cost.

Are evals optional if the loop is small?

No. Small loops still change prompts, models, and tools. Gold-set evals and CI gates are how you know the custom path still works after edits.

When should we abandon custom for a maintained framework?

When roadmap needs outgrow the loop, staffing cannot own reliability, or you are reimplementing generic orchestration weekly. Use the decision cheatsheet in this section.

Custom Agent Runtimes Basics - first hand-rolled loop and dispatcher
Designing a ReAct Loop from Scratch in FastAPI or Express - API-shaped recipe
When a Custom Runtime Stops Being Worth Maintaining - exit checklist
Custom Agent Runtimes Best Practices - maintainability practices
Framework-Free vs Framework-Based: When to Roll Your Own - complementary trade-off frame
What Native Provider Agent Primitives Give You Over a Framework - native SDK alternative

Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana @solana/web3.js + @solana/spl-token.

Why Some Teams Build a Custom Agent Runtime Instead of Adopting a Framework

Search Documentation

Why Some Teams Build a Custom Agent Runtime Instead of Adopting a Framework