Custom Agent Runtimes Best Practices

Ten practices for keeping a custom agent runtime maintainable as it grows - plus operations checks so the loop does not become an accidental unstaffed framework. Use this list in design reviews and before promoting a spike to production.

How to Use This List

• Walk A when designing the loop; B when wiring tools and HTTP; C when operating and evolving.
• Tick an item only when code or runbooks prove it, not as aspiration.
• Pair with When a Custom Runtime Stops Being Worth Maintaining for stay-vs-migrate calls.
• Re-check after new tool plugins, multi-agent profiles, or model gateway changes.

A - Design the Runtime Surface

• 1. Keep domain tools pure. No model SDK imports inside business logic; the loop and adapters own provider types.
• 2. Enforce stop conditions in the host. max_turns, wall-clock timeout, cost caps, and cancel flags are not optional prompt hopes.
• 3. Use a stable stop_reason enum. final_answer | max_turns | timeout | cancelled | tool_policy | model_error | circuit_open (extend carefully).
• 4. Version prompts and schemas as data. Store next to code with owners; do not leave them only in constructor strings on laptops.
• 5. Document why custom exists. ADR with drivers, alternatives, ownership, and exit triggers.

B - Wire Tools, API, and Safety

• 6. Register tools via a plugin registry. Allowlists per agent profile; validate args before side effects.
• 7. Inject auth context from the host. Never trust model-supplied user_id / tenant_id for privileged actions.
• 8. Bound every remote call. Per-tool timeouts, size limits on observations, and circuit breakers on dependencies.
• 9. Expose a thin HTTP/worker API. Validate budgets at the edge; keep the loop importable for evals without the web stack.
• 10. Avoid inventing a private graph framework. If you need checkpoints, joins, or multi-day HITL, adopt a maintained orchestrator instead of agent_os.py.

C - Operate and Evolve

• 11. Trace every production run. run_id, turn, tool names, latencies, stop reasons; redact secrets and PII by default.
• 12. Run an eval harness in CI. Gold cases for core and security tags; gate deploys on thresholds.
• 13. Metric stop_reason and breaker state. Alert on spikes even when HTTP returns 200.
• 14. Pin model and client versions. Verify model ids and tool wire formats at build; config-inject model slugs.
• 15. Review the exit cheatsheet quarterly. Migrate or platformize when B-list pressure items stack up.

Applying These Practices in Order

• Design (1-5): cheapest place to prevent runaway agents and lock-in.
• Wiring (6-10): launch blockers for tool-connected APIs.
• Operate (11-15): keeps custom honest under real traffic and roadmap drift.

Ten Core Practices (title checklist)

1. Pure tools outside the loop.
2. Host-enforced budgets and stops.
3. Stable stop reasons for ops and evals.
4. Prompts/schemas as versioned data.
5. ADR with ownership and exit triggers.
6. Plugin registry + profile allowlists.
7. Host-injected auth context.
8. Timeouts, observation limits, breakers.
9. Eval CI on the same entrypoint as prod.
10. Refuse generic graph reinvention; migrate when needed.

FAQs

Related

• Why Some Teams Build a Custom Agent Runtime Instead of Adopting a Framework - motivation
• Custom Agent Runtimes Basics - first loop
• Designing a ReAct Loop from Scratch in FastAPI or Express - API recipe
• Building an Eval Harness for a Custom Runtime - quality gates
• Circuit Breakers and Observability Without a Third-Party Framework - reliability
• Plugin Architecture for Tools in a Custom Runtime - tool registry
• When a Custom Runtime Stops Being Worth Maintaining - exit cheatsheet

Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana @solana/web3.js + @solana/spl-token.