Custom Agent Runtimes Best Practices

Ten practices for keeping a custom agent runtime maintainable as it grows - plus operations checks so the loop does not become an accidental unstaffed framework. Use this list in design reviews and before promoting a spike to production.

How to Use This List

Walk A when designing the loop; B when wiring tools and HTTP; C when operating and evolving.
Tick an item only when code or runbooks prove it, not as aspiration.
Pair with When a Custom Runtime Stops Being Worth Maintaining for stay-vs-migrate calls.
Re-check after new tool plugins, multi-agent profiles, or model gateway changes.

A - Design the Runtime Surface

1. Keep domain tools pure. No model SDK imports inside business logic; the loop and adapters own provider types.
2. Enforce stop conditions in the host. max_turns, wall-clock timeout, cost caps, and cancel flags are not optional prompt hopes.
3. Use a stable stop_reason enum. final_answer | max_turns | timeout | cancelled | tool_policy | model_error | circuit_open (extend carefully).
4. Version prompts and schemas as data. Store next to code with owners; do not leave them only in constructor strings on laptops.
5. Document why custom exists. ADR with drivers, alternatives, ownership, and exit triggers.

B - Wire Tools, API, and Safety

6. Register tools via a plugin registry. Allowlists per agent profile; validate args before side effects.
7. Inject auth context from the host. Never trust model-supplied user_id / tenant_id for privileged actions.
8. Bound every remote call. Per-tool timeouts, size limits on observations, and circuit breakers on dependencies.
9. Expose a thin HTTP/worker API. Validate budgets at the edge; keep the loop importable for evals without the web stack.
10. Avoid inventing a private graph framework. If you need checkpoints, joins, or multi-day HITL, adopt a maintained orchestrator instead of agent_os.py.

C - Operate and Evolve

11. Trace every production run. run_id, turn, tool names, latencies, stop reasons; redact secrets and PII by default.
12. Run an eval harness in CI. Gold cases for core and security tags; gate deploys on thresholds.
13. Metric stop_reason and breaker state. Alert on spikes even when HTTP returns 200.
14. Pin model and client versions. Verify model ids and tool wire formats at build; config-inject model slugs.
15. Review the exit cheatsheet quarterly. Migrate or platformize when B-list pressure items stack up.

Applying These Practices in Order

Design (1-5): cheapest place to prevent runaway agents and lock-in.
Wiring (6-10): launch blockers for tool-connected APIs.
Operate (11-15): keeps custom honest under real traffic and roadmap drift.

Ten Core Practices (title checklist)

Pure tools outside the loop.
Host-enforced budgets and stops.
Stable stop reasons for ops and evals.
Prompts/schemas as versioned data.
ADR with ownership and exit triggers.
Plugin registry + profile allowlists.
Host-injected auth context.
Timeouts, observation limits, breakers.
Eval CI on the same entrypoint as prod.
Refuse generic graph reinvention; migrate when needed.

FAQs

Why more than ten checkboxes?

Items 1-10 are the core build practices promised in the title. Items 11-15 extend them into operations so the list stays useful after launch.

What is the single highest-leverage practice?

Host-enforced stop conditions plus pure tools. Everything else is easier if those two hold.

Do we need OpenTelemetry on day one?

Structured logs with run_id and stop_reason can start. Add OTel when your platform standard requires it or multi-service joins get hard.

How small should the loop stay?

If generic orchestration (joins, DSLs, checkpoint platforms) dominates, you are past "thin runtime." Re-read the exit cheatsheet.

Should every agent share one process-wide tool catalog?

No. Profile allowlists improve safety and tool selection. Share the registry, not the full allow set.

How do plugins relate to MCP?

Treat MCP as a plugin source that adapts remote tools into the same dispatch contract.

Is a custom runtime compatible with OpenRouter?

Yes. Use the gateway as the model client. Routing policy stays in your config.

What belongs in on-call runbooks?

How to disable an agent route, open/close breakers, raise budget caps carefully, kill runaway workers, and read traces for tool storms.

How often should we re-evaluate staying custom?

At least each quarter or when any durable-resume / multi-team platform need appears.

Can best practices differ for Express vs FastAPI?

Principles match. Language stacks differ for validation and async; reliability rules do not.

Should product managers see this checklist?

Yes for ownership, budgets, eval gates, and exit strategy. Hide wire-format details unless they care.

What is the biggest red flag in code review?

A PR that adds generic workflow engine features "quickly" on top of the loop without an ADR and staffing plan.

Why Some Teams Build a Custom Agent Runtime Instead of Adopting a Framework - motivation
Custom Agent Runtimes Basics - first loop
Designing a ReAct Loop from Scratch in FastAPI or Express - API recipe
Building an Eval Harness for a Custom Runtime - quality gates
Circuit Breakers and Observability Without a Third-Party Framework - reliability
Plugin Architecture for Tools in a Custom Runtime - tool registry
When a Custom Runtime Stops Being Worth Maintaining - exit cheatsheet

Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana @solana/web3.js + @solana/spl-token.

Custom Agent Runtimes Best Practices

How to Use This List

Walk A when designing the loop; B when wiring tools and HTTP; C when operating and evolving.
Tick an item only when code or runbooks prove it, not as aspiration.
Pair with When a Custom Runtime Stops Being Worth Maintaining for stay-vs-migrate calls.
Re-check after new tool plugins, multi-agent profiles, or model gateway changes.

A - Design the Runtime Surface

1. Keep domain tools pure. No model SDK imports inside business logic; the loop and adapters own provider types.
2. Enforce stop conditions in the host. max_turns, wall-clock timeout, cost caps, and cancel flags are not optional prompt hopes.
3. Use a stable stop_reason enum. final_answer | max_turns | timeout | cancelled | tool_policy | model_error | circuit_open (extend carefully).
4. Version prompts and schemas as data. Store next to code with owners; do not leave them only in constructor strings on laptops.
5. Document why custom exists. ADR with drivers, alternatives, ownership, and exit triggers.

B - Wire Tools, API, and Safety

6. Register tools via a plugin registry. Allowlists per agent profile; validate args before side effects.
7. Inject auth context from the host. Never trust model-supplied user_id / tenant_id for privileged actions.
8. Bound every remote call. Per-tool timeouts, size limits on observations, and circuit breakers on dependencies.
9. Expose a thin HTTP/worker API. Validate budgets at the edge; keep the loop importable for evals without the web stack.
10. Avoid inventing a private graph framework. If you need checkpoints, joins, or multi-day HITL, adopt a maintained orchestrator instead of agent_os.py.

C - Operate and Evolve

11. Trace every production run. run_id, turn, tool names, latencies, stop reasons; redact secrets and PII by default.
12. Run an eval harness in CI. Gold cases for core and security tags; gate deploys on thresholds.
13. Metric stop_reason and breaker state. Alert on spikes even when HTTP returns 200.
14. Pin model and client versions. Verify model ids and tool wire formats at build; config-inject model slugs.
15. Review the exit cheatsheet quarterly. Migrate or platformize when B-list pressure items stack up.

Applying These Practices in Order

Design (1-5): cheapest place to prevent runaway agents and lock-in.
Wiring (6-10): launch blockers for tool-connected APIs.
Operate (11-15): keeps custom honest under real traffic and roadmap drift.

Ten Core Practices (title checklist)

Pure tools outside the loop.
Host-enforced budgets and stops.
Stable stop reasons for ops and evals.
Prompts/schemas as versioned data.
ADR with ownership and exit triggers.
Plugin registry + profile allowlists.
Host-injected auth context.
Timeouts, observation limits, breakers.
Eval CI on the same entrypoint as prod.
Refuse generic graph reinvention; migrate when needed.

FAQs

Why more than ten checkboxes?

Items 1-10 are the core build practices promised in the title. Items 11-15 extend them into operations so the list stays useful after launch.

What is the single highest-leverage practice?

Host-enforced stop conditions plus pure tools. Everything else is easier if those two hold.

Do we need OpenTelemetry on day one?

Structured logs with run_id and stop_reason can start. Add OTel when your platform standard requires it or multi-service joins get hard.

How small should the loop stay?

If generic orchestration (joins, DSLs, checkpoint platforms) dominates, you are past "thin runtime." Re-read the exit cheatsheet.

Should every agent share one process-wide tool catalog?

No. Profile allowlists improve safety and tool selection. Share the registry, not the full allow set.

How do plugins relate to MCP?

Treat MCP as a plugin source that adapts remote tools into the same dispatch contract.

Is a custom runtime compatible with OpenRouter?

Yes. Use the gateway as the model client. Routing policy stays in your config.

What belongs in on-call runbooks?

How to disable an agent route, open/close breakers, raise budget caps carefully, kill runaway workers, and read traces for tool storms.

How often should we re-evaluate staying custom?

At least each quarter or when any durable-resume / multi-team platform need appears.

Can best practices differ for Express vs FastAPI?

Principles match. Language stacks differ for validation and async; reliability rules do not.

Should product managers see this checklist?

Yes for ownership, budgets, eval gates, and exit strategy. Hide wire-format details unless they care.

What is the biggest red flag in code review?

A PR that adds generic workflow engine features "quickly" on top of the loop without an ADR and staffing plan.

Why Some Teams Build a Custom Agent Runtime Instead of Adopting a Framework - motivation
Custom Agent Runtimes Basics - first loop
Designing a ReAct Loop from Scratch in FastAPI or Express - API recipe
Building an Eval Harness for a Custom Runtime - quality gates
Circuit Breakers and Observability Without a Third-Party Framework - reliability
Plugin Architecture for Tools in a Custom Runtime - tool registry
When a Custom Runtime Stops Being Worth Maintaining - exit cheatsheet

Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana @solana/web3.js + @solana/spl-token.

Custom Agent Runtimes Best Practices

Buscar en la documentación

Custom Agent Runtimes Best Practices