LLMs Behind Agents Best Practices

Ten practices for selecting, budgeting, and swapping models so the "brain" layer of your agent stays cheap enough to run, strong enough to finish, and flexible enough to change.

How to Use This List

Use it as a checklist when designing a new agent or reviewing an existing one.
Apply selection and context practices before optimizing micro-prompt wording.
Treat cost and swap practices as production readiness, not polish.
Revisit after major model releases or pricing changes (verify current rates and limits at build).

A - Selection

1. Pick for tool-calling reliability on your tools, not for chat demos. An agent brain that cannot emit valid tool calls is the wrong model no matter how fluent it is in a playground.
2. Default to a solid mid-tier model for the main loop. Reserve frontier tiers for hard planning, tough coding, or ambiguous judgment where mid-tier traces show systematic failure.
3. Route high-volume simple steps to small, cheap, or local models. Classification, extraction, and triage should not inherit frontier pricing by accident.
4. Verify context window, tool support, and pricing before integration. Read the model card as an engineering constraint set, not marketing copy.

B - Context and Cost

5. Budget turns x re-sent input tokens, not only final answers. Multi-turn tool loops re-bill history; design for that multiplicative cost from day one.
6. Keep tool catalogs and tool results small. Extra schemas and fat payloads waste context, confuse selection, and inflate every subsequent call.
7. Put hard ceilings on turns, wall-clock time, and spend. Unbounded autonomy is an unbounded invoice and an unbounded failure mode.

C - Swapping and Operations

8. Load model ids from config and isolate provider adapters. Business logic should not hardcode vendor SDKs in every node if you want runtime flexibility.
9. Log model id, tokens, and step outcomes on every call. You cannot improve routing, budgets, or failover without attribution.
10. Promote model changes through evals and allowlists. New defaults should pass tool-use and golden-task suites before they own production traffic.

Applying the Practices

Phase	Practices to emphasize
Prototype	1, 2, 4, 7
First production path	5, 6, 8, 9
Scale and multi-model	3, 10, plus tighter routing from 2-3
Incident or cost spike	5, 7, 9, then revisit 2-3

FAQs

What is the single highest-leverage practice?

Measuring real multi-turn token usage early (practice 5 + 9), because it forces honest model and context design before habits calcify.

Should every team multi-home across providers immediately?

No.

Ship one reliable path with a clean adapter, then add a second model for failover or cost routing when metrics justify the complexity.

How do these practices relate to Ollama or local models?

Local models are one way to execute practices 3 and parts of privacy-sensitive deployments.

They still need practices 1, 6, 7, and 10 - local does not excuse weak tool calling or unbounded loops.

Is "always use the smartest model" ever right?

Rarely as a blanket default.

It can be right for a narrow, high-stakes step with low volume, especially behind human approval for side effects.

How often should we revisit model defaults?

After material model launches, pricing changes, or dashboard regressions - and on a regular cadence (for example quarterly) even if nothing feels broken.

Do frameworks remove the need for these practices?

No.

Frameworks accelerate wiring; they do not set your ceilings, evals, or routing policy.

What belongs in the allowlist for production models?

Model ids that passed evals, have known cost classes, clear owners, and documented capabilities (tools, vision, context).

How small should tool results be?

Small enough that the next decision is obvious without re-reading noise - prefer filtered fields and summaries with links to full artifacts.

Where should max-turn limits live?

In the agent runtime configuration, enforced server-side, not only as a polite suggestion in the prompt.

Can prompt caching replace context discipline?

No.

Caching can reduce the cost of stable prefixes when available; it does not fix irrelevant tool dumps or runaway turns.

How do we know a cheaper model is "good enough"?

When success rate, constraint adherence, and human edit distance meet the bar on a fixed eval set at acceptable turn counts - not when a single happy demo works.

What is a common failure mode when swapping models?

Assuming prompts and tool schemas are fully portable without re-running evals.

Re-validate tool-calling and multi-turn constraint retention on every new default.

How an LLM's Context Window Bounds What an Agent Can Do - context as the hard ceiling behind practices 5-6
Tokens, Pricing, and Why Agent Costs Add Up Fast - economics behind practices 5 and 7
Matching Model Tier to Agent Task Complexity - decision aid for practices 2-3
Why Model Choice Is a Runtime Decision, Not a One-Time One - architecture behind practices 8-10
LLMs Behind Agents Basics - starter walkthrough for first-time model selection

Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana @solana/web3.js + @solana/spl-token.

LLMs Behind Agents Best Practices

Ten practices for selecting, budgeting, and swapping models so the "brain" layer of your agent stays cheap enough to run, strong enough to finish, and flexible enough to change.

How to Use This List

Use it as a checklist when designing a new agent or reviewing an existing one.
Apply selection and context practices before optimizing micro-prompt wording.
Treat cost and swap practices as production readiness, not polish.
Revisit after major model releases or pricing changes (verify current rates and limits at build).

A - Selection

1. Pick for tool-calling reliability on your tools, not for chat demos. An agent brain that cannot emit valid tool calls is the wrong model no matter how fluent it is in a playground.
2. Default to a solid mid-tier model for the main loop. Reserve frontier tiers for hard planning, tough coding, or ambiguous judgment where mid-tier traces show systematic failure.
3. Route high-volume simple steps to small, cheap, or local models. Classification, extraction, and triage should not inherit frontier pricing by accident.
4. Verify context window, tool support, and pricing before integration. Read the model card as an engineering constraint set, not marketing copy.

B - Context and Cost

5. Budget turns x re-sent input tokens, not only final answers. Multi-turn tool loops re-bill history; design for that multiplicative cost from day one.
6. Keep tool catalogs and tool results small. Extra schemas and fat payloads waste context, confuse selection, and inflate every subsequent call.
7. Put hard ceilings on turns, wall-clock time, and spend. Unbounded autonomy is an unbounded invoice and an unbounded failure mode.

C - Swapping and Operations

8. Load model ids from config and isolate provider adapters. Business logic should not hardcode vendor SDKs in every node if you want runtime flexibility.
9. Log model id, tokens, and step outcomes on every call. You cannot improve routing, budgets, or failover without attribution.
10. Promote model changes through evals and allowlists. New defaults should pass tool-use and golden-task suites before they own production traffic.

Applying the Practices

Phase	Practices to emphasize
Prototype	1, 2, 4, 7
First production path	5, 6, 8, 9
Scale and multi-model	3, 10, plus tighter routing from 2-3
Incident or cost spike	5, 7, 9, then revisit 2-3

FAQs

What is the single highest-leverage practice?

Measuring real multi-turn token usage early (practice 5 + 9), because it forces honest model and context design before habits calcify.

Should every team multi-home across providers immediately?

No.

Ship one reliable path with a clean adapter, then add a second model for failover or cost routing when metrics justify the complexity.

How do these practices relate to Ollama or local models?

Local models are one way to execute practices 3 and parts of privacy-sensitive deployments.

They still need practices 1, 6, 7, and 10 - local does not excuse weak tool calling or unbounded loops.

Is "always use the smartest model" ever right?

Rarely as a blanket default.

It can be right for a narrow, high-stakes step with low volume, especially behind human approval for side effects.

How often should we revisit model defaults?

After material model launches, pricing changes, or dashboard regressions - and on a regular cadence (for example quarterly) even if nothing feels broken.

Do frameworks remove the need for these practices?

No.

Frameworks accelerate wiring; they do not set your ceilings, evals, or routing policy.

What belongs in the allowlist for production models?

Model ids that passed evals, have known cost classes, clear owners, and documented capabilities (tools, vision, context).

How small should tool results be?

Small enough that the next decision is obvious without re-reading noise - prefer filtered fields and summaries with links to full artifacts.

Where should max-turn limits live?

In the agent runtime configuration, enforced server-side, not only as a polite suggestion in the prompt.

Can prompt caching replace context discipline?

No.

Caching can reduce the cost of stable prefixes when available; it does not fix irrelevant tool dumps or runaway turns.

How do we know a cheaper model is "good enough"?

When success rate, constraint adherence, and human edit distance meet the bar on a fixed eval set at acceptable turn counts - not when a single happy demo works.

What is a common failure mode when swapping models?

Assuming prompts and tool schemas are fully portable without re-running evals.

Re-validate tool-calling and multi-turn constraint retention on every new default.

How an LLM's Context Window Bounds What an Agent Can Do - context as the hard ceiling behind practices 5-6
Tokens, Pricing, and Why Agent Costs Add Up Fast - economics behind practices 5 and 7
Matching Model Tier to Agent Task Complexity - decision aid for practices 2-3
Why Model Choice Is a Runtime Decision, Not a One-Time One - architecture behind practices 8-10
LLMs Behind Agents Basics - starter walkthrough for first-time model selection

Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana @solana/web3.js + @solana/spl-token.

LLMs Behind Agents Best Practices

Search Documentation

LLMs Behind Agents Best Practices