Matching Model Tier to Agent Task Complexity
Most agent systems waste money on frontier models for easy steps, or waste human time by underpowering hard steps.
This cheatsheet maps task complexity to model tier so routing is deliberate.
Exact model names and prices change; treat tiers as roles and pin current ids in config (verify at build).
How to Use This List
- Classify the step, not the whole product: one agent workflow often needs multiple tiers.
- Start with the recommended tier, then escalate only when evals or production traces show systematic failure.
- Prefer changing the model for a step before endlessly elongating prompts.
- Revisit routing after model launches; yesterday's "frontier-only" step may be mid-tier today.
Tier Roles (Portable Labels)
| Tier role | Typical profile | Agent use |
|---|---|---|
| Small / cheap | Fast, low $ per token, lighter reasoning | Classification, extraction, simple rewrites, triage |
| Mid / default | Balanced quality, cost, latency | Main loop for many product agents |
| Frontier | Strongest reasoning/coding/tool use you can buy | Hard planning, ambiguous judgment, tough coding |
| Local / self-hosted | Privacy and marginal-cost control; variable quality | Private data paths, offline, bulk cheap steps |
| Specialized | Embeddings, rerankers, ASR, etc. | Retrieval and non-chat utilities - not the agent brain |
Decision 1: Label or Route an Incoming Item
Scenario: Classify a ticket, email, or event into a small label set before any heavy work.
| Rank | Choice | Approach |
|---|---|---|
| Best | Small / cheap | Fast structured label; escalate only on low confidence |
| 2nd | Mid / default | If labels are subtle or policy-sensitive |
| 3rd | Frontier | Rarely justified for pure taxonomy work |
Wrong choice: Running every triage call on a frontier model "to be safe." Why best is best: The step is high volume and low ambiguity relative to later work; savings compound immediately.
Decision 2: Extract Structured Fields
Scenario: Pull dates, amounts, ids, or JSON fields from semi-structured text.
| Rank | Choice | Approach |
|---|---|---|
| Best | Small / cheap or mid | Use strict schemas; validate deterministically after the call |
| 2nd | Mid / default | When documents are messy or domain language is dense |
| 3rd | Frontier | Only if extraction needs deep inference across conflicting evidence |
Wrong choice: Skipping schema validation because a frontier model "usually gets JSON right." Why best is best: Validation + cheaper models usually beats premium rates for field extraction.
Decision 3: Single-Tool Fact Lookup
Scenario: Call one search or DB tool, then answer.
| Rank | Choice | Approach |
|---|---|---|
| Best | Mid / default | Reliable tool selection without frontier cost |
| 2nd | Small / cheap | If the tool choice is obvious and args are simple |
| 3rd | Frontier | When query formation is hard or sources conflict |
Wrong choice: Assuming any model that can chat can form correct tool arguments under production schemas. Why best is best: Tool reliability matters more than peak creativity for this shape.
Decision 4: Multi-Step Tool Loop (General Agent)
Scenario: Several tools, branching decisions, 5-15 turns.
| Rank | Choice | Approach |
|---|---|---|
| Best | Mid / default | Default brain for the loop; monitor turn counts |
| 2nd | Frontier | If mid-tier loops thrash or miss constraints |
| 3rd | Small / cheap | Only for highly constrained, narrow workflows |
Wrong choice: Small model with a huge tool catalog and no turn cap. Why best is best: Mid-tier usually balances tool use quality with the cost of multi-turn re-billing.
Decision 5: Hard Planning or Ambiguous Goals
Scenario: Vague goals, trade-offs, multi-constraint planning before execution.
| Rank | Choice | Approach |
|---|---|---|
| Best | Frontier | Produce the plan; optionally execute steps on cheaper tiers |
| 2nd | Mid / default | Acceptable for narrower planning domains |
| 3rd | Small / cheap | High risk of shallow or contradictory plans |
Wrong choice: Generating a bad plan cheaply, then spending many execution turns recovering. Why best is best: Planning mistakes are expensive multipliers on every later step.
Decision 6: Coding, Patches, and Repo Reasoning
Scenario: Multi-file edits, nontrivial debugging, or test-driven fixes.
| Rank | Choice | Approach |
|---|---|---|
| Best | Frontier or strong mid coding model | Pick based on current coding evals for your stack |
| 2nd | Mid / default | Routine edits with good tests as a safety net |
| 3rd | Small / cheap | Tiny, local, well-specified transforms only |
Wrong choice: Cheapest model on an untested multi-file refactor. Why best is best: Failed coding loops burn tokens and engineer time together.
Decision 7: Long-Document Synthesis
Scenario: Summarize or compare long materials with retrieval or large context.
| Rank | Choice | Approach |
|---|---|---|
| Best | Mid / default with retrieval discipline | Cap chunks; cite sources |
| 2nd | Frontier long-context | When cross-document reasoning is genuinely hard |
| 3rd | Small / cheap | Fine for map-reduce map steps, weak for final synthesis if subtle |
Wrong choice: Stuffing the full corpus into a frontier window "because we can." Why best is best: Context packing and map-reduce often beat brute-force long context on cost and focus.
Decision 8: High-Stakes User-Facing Judgment
Scenario: Refunds, legal-ish language, medical-adjacent triage support, financial advice adjacent tasks.
| Rank | Choice | Approach |
|---|---|---|
| Best | Frontier + human-in-the-loop | Model assists; policy gates irreversible actions |
| 2nd | Mid / default + stricter tools/policies | When domain is narrow and tools enforce rules |
| 3rd | Small / cheap unsupervised | Inappropriate for unsupervised high stakes |
Wrong choice: Equating "higher model tier" with "safe to act autonomously." Why best is best: Stakes demand oversight and controls; tier alone is not a control.
Decision 9: Bulk Offline Processing
Scenario: Nightly enrichment of thousands of records.
| Rank | Choice | Approach |
|---|---|---|
| Best | Small / cheap or local | Batch-friendly; strict schemas; sample QA |
| 2nd | Mid / default | If error cost per record is high |
| 3rd | Frontier | Sample-only escalation for edge clusters |
Wrong choice: Frontier rates on every record without a sampling strategy. Why best is best: Volume makes unit economics dominate.
Decision 10: When to Escalate Mid-Run
Scenario: The default model is stuck: repeated tools, low confidence, or policy flags.
| Rank | Choice | Approach |
|---|---|---|
| Best | Escalation policy | One retry with repair, then mid→frontier or human |
| 2nd | Restart with narrower plan | Compress context; reset tools; try again once |
| 3rd | Endless same-tier retries | Usually burns money without new information |
Wrong choice: Quietly looping until max tokens with no escalation signal. Why best is best: Escalation is a product feature, not an admission of defeat.
Applying the List in Order
- Separate steps (1-3, 9): route volume and structure work downward.
- Protect the main loop (4, 6): pick a default that tool-calls reliably; measure turns.
- Spend up for leverage (5, 7, 8): planning, hard coding, and high stakes justify premium tiers or humans.
- Encode escalation (10): never rely on hope as the only recovery path.
Quick Routing Table
| Signal in the step | Bias toward |
|---|---|
| High volume, short output, strict schema | Small / cheap / local |
| Multi-tool, moderate ambiguity | Mid / default |
| Open-ended planning, hard coding, conflicting evidence | Frontier |
| Private raw data, offline requirement | Local first |
| Irreversible external side effects | Any tier + human/tool gates |
FAQs
Should an entire agent use one model?
Only for early prototypes.
Production systems almost always benefit from at least cheap-vs-default splitting.
How do I know a step is "too hard" for a cheaper model?
Look for elevated tool-error rates, turn counts, constraint violations, or human edit distance - not vibes from one demo.
Is frontier always better for tool calling?
Often stronger, not always worth it.
If a mid-tier already selects tools correctly on your eval set, paying frontier rates for that step is optional.
Where do local models fit in the tier ladder?
As a parallel axis: privacy and marginal cost, with a quality ceiling you must measure.
They can play "small/cheap" or even "mid" roles when evals support it.
Can routing logic itself be an LLM call?
Yes, but keep it cheap and constrained.
A tiny classifier or rules engine is often enough to pick the tier for the next step.
What should I log to improve routing?
Model id, step name, tokens, turns, tool error rate, and final success.
Without those, routing debates stay anecdotal.
Does a larger context window mean a higher tier?
No.
Context size and reasoning strength are related in product packaging but not identical; pick for the bottleneck you actually have.
How often should routing rules change?
When evals shift after a model release, or when cost/quality dashboards show a step is mis-tiered.
Avoid weekly thrash without evidence.
What is the biggest anti-pattern?
"Best model for everything" in production multi-turn agents - it maximizes spend and hides design problems behind raw capability.
What is the second biggest anti-pattern?
"Cheapest model for everything" without escalation - it maximizes silent failure and human cleanup.
Should embeddings use the agent brain model?
No.
Use embedding models for embeddings; keep chat/completion models for decisions and generation.
How does this interact with OpenRouter-style gateways?
Gateways make tier swaps easy because model ids become configuration.
You still need the routing policy and evals; the gateway does not invent them.
Related
- Tokens, Pricing, and Why Agent Costs Add Up Fast - why multi-turn economics punish wrong tiers
- Major LLM Providers for Agent Building: OpenAI, Anthropic, Google, xAI - where hosted tiers come from
- Local & Self-Hosted Models with Ollama for Agent Workloads - when local plays the cheap/private role
- Why Model Choice Is a Runtime Decision, Not a One-Time One - implementing swaps cleanly
- LLMs Behind Agents Best Practices - habits for selection across an agent's lifetime
Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana
@solana/web3.js+@solana/spl-token.