- A provider API key (OpenAI, Anthropic, Google, xAI, or a gateway such as OpenRouter) or a local runtime such as Ollama.
- Comfort reading a model card for context window size, tool-use support, and pricing units (usually per million tokens).
- A rough idea of one agent task you want to power (for example: "answer with a search tool" or "summarize a file").
Before wiring an agent loop, open the model docs and answer four questions in order.
- What is the context window (input budget)?
- Does the model support native tool / function calling?
- What are input and output prices per million tokens (verify current rates at build)?
- What latency profile is expected for interactive use?
- Context window bounds how much history, tools, and retrieval you can pack per turn.
- Tool-calling support is nearly mandatory for real agents; pure text-only models force brittle parsing.
- Pricing is almost always split into input vs output, and multi-turn agents pay input repeatedly.
- Latency matters because agent loops may call the model many times before the user sees a result.
Related: How an LLM's Context Window Bounds What an Agent Can Do - why the token budget is the agent's hard ceiling
Pick one solid mid-tier model as your default while you learn the loop.
Do not start by optimizing for "smartest available."
- Prefer a model with stable function calling, decent coding or reasoning quality, and published pricing you understand.
- Leave room to swap models later; hardcode the model name in one config place only.
- Use the same default for the first vertical slice: one goal, one or two tools, a max-turn limit.
- Escalate to a heavier frontier model only after you see the prototype fail for capability reasons, not for prompt-wiring reasons.
Sketch the token budget on paper before the first long run.
Example agent: system prompt + two tools + user goal + up to five tool results.
- Estimate system + tool schemas (often hundreds to a few thousand tokens).
- Estimate one tool result (for example, a search page or file chunk).
- Multiply tool results by expected turns.
- Leave headroom for the model's next reply and any reasoning tokens.
- If the rough sum already approaches the window, redesign before you code more tools.
- Large tool dumps are the usual surprise, not the user message.
- Headroom is part of the design; filling the window to 100% leaves no room to answer.
- Re-measure with the provider tokenizer on a real transcript after the first successful run.
Not every model string that "chats" is a good agent brain.
When comparing OpenAI, Anthropic, Google, xAI, or open-weight models, verify:
- Native structured tool / function calling (not only free-form "use a tool" prose).
- Parallel tool calls if your agent needs multiple lookups in one step.
- Structured outputs or JSON mode if you need strict schemas outside tools.
- Any provider-specific quirks (tool result message roles, max tools, streaming tool calls).
Related: Function Calling & Tool Use Support Across Providers - a side-by-side reference for agent builders
A single chat message and a ten-step agent run are different products priced in the same units.
Example mental math (use placeholders; verify rates at build):
-
One agent turn may re-send 4,000 input tokens of history and emit 200 output tokens.
-
Ten turns re-bill much of that history each time, so input tokens dominate.
-
A cheaper model with more retries can still cost more than a stronger model that finishes in fewer turns.
-
Always estimate turns x average input size, not just final answer length.
-
Output is usually priced higher per token than input, but agents often spend more on repeated input.
-
Cap max turns early so cost has a ceiling while you learn.
-
Log token usage per run from day one.
Once the prototype works, split work by difficulty instead of one model for everything.
Example workflow: inbox triage agent.
- Use a small / cheap model to classify message type and urgency.
- Use a mid-tier model to draft a reply when the class is routine.
- Escalate ambiguous or high-stakes messages to a frontier model or a human.
- Classification and extraction are classic cheap-model jobs.
- Multi-step planning, coding, and ambiguous judgment often need a stronger tier.
- Routing by task shape usually beats "always use the best model."
- Keep the same tool interface across tiers so escalation is a model swap, not a rewrite.
Related: Matching Model Tier to Agent Task Complexity - a decision checklist for cheap vs frontier routing
Treat the model id as runtime configuration, not a constant buried in business logic.
Example setup for a Python agent skeleton:
import os
MODEL_ID = os.getenv("AGENT_MODEL", "provider/mid-tier-model")
MAX_TURNS = int(os.getenv("AGENT_MAX_TURNS", "8"))
- Load
MODEL_ID from env or a config file.
- Pass that string into your SDK / gateway client in one place.
- Keep tool definitions and loop logic independent of the model vendor when possible.
- Add a second env for a fallback model when the primary is rate-limited or down.
- Gateways such as OpenRouter make model swaps mostly a string change across many providers.
- Direct provider SDKs may need thin adapters for message and tool formats.
- Runtime model choice is how you respond to price changes, outages, and new releases without redesigning the agent.
- Pair this with evals so a swap is a measured decision, not a guess.
Related: Why Model Choice Is a Runtime Decision, Not a One-Time One - the architecture case for swappable models
Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana @solana/web3.js + @solana/spl-token.