A Decision Framework: Is This Problem a Good Fit for an Agent?
Use this ranked decision guide when someone proposes "let's make an agent for X."
//
Search across all documentation pages
Use this ranked decision guide when someone proposes "let's make an agent for X."
Walk the decisions in order.
Earlier "wrong choices" are cheaper to reverse than late architecture commits.
Scenario: Stakeholders want an agent, but success is "looks good to the VP."
| Rank | Choice | Approach |
|---|---|---|
| Best | Machine-checkable or cheaply sampleable outcome | Tests pass, fields updated, N citations, SLA met |
| 2nd | Structured rubric + spot checks | Scorecards on a fixed eval set |
| 3rd | Human rating only on drafts | Copilot path with accept/reject metrics |
Wrong choice: Ship unsupervised autonomy when only senior experts can judge quality.
Why best is best: Without a checkable outcome, you cannot know if the loop helps or hallucinates confidently.
Scenario: The process can be drawn as a fixed flowchart today.
| Rank | Choice | Approach |
|---|---|---|
| Best | Script / workflow engine if path is known | Deterministic steps, LLM only inside fixed slots |
| 2nd | Bounded agent with small tool allowlist | Model chooses among few steps when rare branches appear |
| 3rd | Open agent loop | Only when branching is frequent and unpredictable |
Wrong choice: Force a free-form agent onto a stable RPA-style path.
Why best is best: Known paths are cheaper to test, operate, and explain than multi-turn tool loops.
Scenario: The agent needs live systems to act or observe.
| Rank | Choice | Approach |
|---|---|---|
| Best | Stable APIs with auth and dry-run modes | Read tools first, then gated writes |
| 2nd | Thin adapters over existing CLIs/RPA | Accept fragility; limit blast radius |
| 3rd | Human-provided tool results (copilot) | Agent plans; human pastes outcomes |
Wrong choice: Promise an agent while the "tools" are still slideware integrations.
Why best is best: Agency without an action surface is just chat with extra latency.
Scenario: The agent might email customers, move money, merge code, or change production.
| Rank | Choice | Approach |
|---|---|---|
| Best | Draft-only or reversible actions first | PR not push; suggest not send; queue not execute |
| 2nd | Human approval on irreversible tools | Step-up review with full trace |
| 3rd | Autonomous writes with hard limits | Caps, canaries, instant kill switch |
Wrong choice: Full write access on day one because the demo felt smart.
Why best is best: Risk scales with irreversibility and blast radius, not with model marketing tiers.
Scenario: Ops asks what the agent did last Tuesday.
| Rank | Choice | Approach |
|---|---|---|
| Best | Full decide-act-observe trace with redaction | Tool names, args, results, stop reason |
| 2nd | Sampled traces + outcome logs | Accept gaps only on low-risk paths |
| 3rd | Final answer only | Accept only for pure draft copilots |
Wrong choice: Production autonomy without inspectable history.
Why best is best: You cannot improve or defend a system you cannot reconstruct.
Scenario: The ticket mixes research language with "just update the system."
| Rank | Choice | Approach |
|---|---|---|
| Best | Name one primary shape | Research, execution, or monitoring |
| 2nd | Explicit multi-stage pipeline | Separate agents/workflows per stage |
| 3rd | One mega-agent for all stages | Only with strong routing and policy |
Wrong choice: "Industry agent" with no shape, no stages, unlimited tools.
Why best is best: Shape predicts tools, context growth, and metrics better than vertical labels.
Scenario: Leadership wants "fully autonomous" in the press release.
| Rank | Choice | Approach |
|---|---|---|
| Best | Lowest level that clears the goal | Single-turn tool → multi-turn → supervised goal |
| 2nd | Supervised goal agent with write gates | Level 3 with approvals |
| 3rd | Long-running / exception-only | Only after eval maturity |
Wrong choice: Jump to unattended autonomy to match competitor marketing.
Why best is best: Extra freedom often reduces reliability until evals and policy catch up.
Scenario: The user experience needs a sub-second answer on a mobile screen.
| Rank | Choice | Approach |
|---|---|---|
| Best | Single completion or one tool call | Cache, classify, retrieve once |
| 2nd | Async agent job with progress UI | User is not blocked on every turn |
| 3rd | Synchronous multi-turn agent | Only when wait is acceptable |
Wrong choice: A 12-tool loop behind a "instant answer" button.
Why best is best: Agents multiply tokens and wall time; product constraints must win.
Scenario: The loop hits max turns or low confidence.
| Rank | Choice | Approach |
|---|---|---|
| Best | Named human queue or on-call path | Ticket with trace link |
| 2nd | Safe degrade to draft / no-op | User completes manually |
| 3rd | Retry blindly | Only with strict budgets |
Wrong choice: Silent failure or endless retries with no owner.
Why best is best: Production agents are ops systems; ownership is part of fit.
Scenario: A shiny demo on three cherry-picked prompts.
| Rank | Choice | Approach |
|---|---|---|
| Best | ≥20-50 golden tasks with expected outcomes | Track success, cost, interventions |
| 2nd | Small golden set + weekly expansion | Explicit coverage gaps |
| 3rd | Demo-only validation | Spike, not launch |
Wrong choice: Scale traffic on vibes and executive screenshots.
Why best is best: Fit is empirical. Without tasks, "good fit" is a slogan.
Scenario: After Decisions 1-10, several "Best" rows pointed away from agents.
| Rank | Choice | Approach |
|---|---|---|
| Best | Ship the simpler product | Script, workflow, RAG assistant, copilot |
| 2nd | Narrow agent slice beside the simple path | Route only hard cases to the loop |
| 3rd | Full agent rewrite of a working system | Rare; needs clear ROI |
Wrong choice: Rebuild a reliable workflow as an agent for branding.
Why best is best: The framework's job is to protect value, not to force the word "agent."
| Stage | Decisions | Exit criterion |
|---|---|---|
| Problem truth | 1-2 | Verifiable goal; path known vs branching |
| Feasibility | 3-5 | Tools, risk gates, traces |
| Design | 6-8 | Shape, autonomy, latency/cost |
| Launch | 9-11 | Owner, evals, honest non-agent option |
If any mandatory Best path is blocked, default to NO-GO as an agent and pick the 2nd-best product shape instead.
Decisions 1, 3, 4, 5, 9, and 10 are near-mandatory for production autonomy. Others can accept 2nd place with documented risk.
A serious first pass fits in 60-90 minutes with eng + product + an owner for risk. Deep spikes follow only on GO or GO-with-caveats.
Still run the framework. Ship the highest-ranked feasible choice (often a gated copilot) and keep the public name if you must, but do not expand write tools to match the press release.
No. Model quality does not create APIs, auth, or dry-run modes.
After Decisions 6-7. Multi-agent is an architecture choice, not a substitute for fit, tools, or evals.
Model evals score capability. This framework scores whether an agent system should exist for the problem at all.
A one-page brief: goal, decisions 1-11 picks, tool list, autonomy level, eval set size, escalation owner.
No. Copilot is often the correct product when judgment is subjective or stakes are high.
After major tool changes, incidents, or quarterly portfolio reviews. Autonomy should be allowed to decrease.
Path is actually known (should be a workflow), but the team wants agent branding and skips evals.
They are the red-flag cheatsheet version of failed decisions here (especially 3, 4, 8, 10).
Yes if one named person owns escalations and the agent fails closed. "Nobody" is not an owner.
Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana
@solana/web3.js+@solana/spl-token.
Reviewed by Chris St. John·Last updated Jul 16, 2026