A Decision Framework: Is This Problem a Good Fit for an Agent?
Use this ranked decision guide when someone proposes "let's make an agent for X."
Walk the decisions in order.
Earlier "wrong choices" are cheaper to reverse than late architecture commits.
How to Use This List
- Answer each decision with current evidence, not future hopes.
- Record Best / 2nd / Wrong choice in a one-page brief.
- Stop early if a decision says "build a script or copilot instead."
- Re-run after a spike when tools or evals change.
- Treat write-path and irreversible actions as separate decisions even if the demo is read-only.
Decision 1: Is the goal verifiable without a senior expert every time?
Scenario: Stakeholders want an agent, but success is "looks good to the VP."
| Rank | Choice | Approach |
|---|---|---|
| Best | Machine-checkable or cheaply sampleable outcome | Tests pass, fields updated, N citations, SLA met |
| 2nd | Structured rubric + spot checks | Scorecards on a fixed eval set |
| 3rd | Human rating only on drafts | Copilot path with accept/reject metrics |
Wrong choice: Ship unsupervised autonomy when only senior experts can judge quality.
Why best is best: Without a checkable outcome, you cannot know if the loop helps or hallucinates confidently.
Decision 2: Is the path fully known, or does it branch on intermediate results?
Scenario: The process can be drawn as a fixed flowchart today.
| Rank | Choice | Approach |
|---|---|---|
| Best | Script / workflow engine if path is known | Deterministic steps, LLM only inside fixed slots |
| 2nd | Bounded agent with small tool allowlist | Model chooses among few steps when rare branches appear |
| 3rd | Open agent loop | Only when branching is frequent and unpredictable |
Wrong choice: Force a free-form agent onto a stable RPA-style path.
Why best is best: Known paths are cheaper to test, operate, and explain than multi-turn tool loops.
Decision 3: Do the required tools exist (or can they in days, not months)?
Scenario: The agent needs live systems to act or observe.
| Rank | Choice | Approach |
|---|---|---|
| Best | Stable APIs with auth and dry-run modes | Read tools first, then gated writes |
| 2nd | Thin adapters over existing CLIs/RPA | Accept fragility; limit blast radius |
| 3rd | Human-provided tool results (copilot) | Agent plans; human pastes outcomes |
Wrong choice: Promise an agent while the "tools" are still slideware integrations.
Why best is best: Agency without an action surface is just chat with extra latency.
Decision 4: What is the cost of a wrong action?
Scenario: The agent might email customers, move money, merge code, or change production.
| Rank | Choice | Approach |
|---|---|---|
| Best | Draft-only or reversible actions first | PR not push; suggest not send; queue not execute |
| 2nd | Human approval on irreversible tools | Step-up review with full trace |
| 3rd | Autonomous writes with hard limits | Caps, canaries, instant kill switch |
Wrong choice: Full write access on day one because the demo felt smart.
Why best is best: Risk scales with irreversibility and blast radius, not with model marketing tiers.
Decision 5: Can you run and review traces end to end?
Scenario: Ops asks what the agent did last Tuesday.
| Rank | Choice | Approach |
|---|---|---|
| Best | Full decide-act-observe trace with redaction | Tool names, args, results, stop reason |
| 2nd | Sampled traces + outcome logs | Accept gaps only on low-risk paths |
| 3rd | Final answer only | Accept only for pure draft copilots |
Wrong choice: Production autonomy without inspectable history.
Why best is best: You cannot improve or defend a system you cannot reconstruct.
Decision 6: Which task shape is primary?
Scenario: The ticket mixes research language with "just update the system."
| Rank | Choice | Approach |
|---|---|---|
| Best | Name one primary shape | Research, execution, or monitoring |
| 2nd | Explicit multi-stage pipeline | Separate agents/workflows per stage |
| 3rd | One mega-agent for all stages | Only with strong routing and policy |
Wrong choice: "Industry agent" with no shape, no stages, unlimited tools.
Why best is best: Shape predicts tools, context growth, and metrics better than vertical labels.
Decision 7: What autonomy level is the minimum that still works?
Scenario: Leadership wants "fully autonomous" in the press release.
| Rank | Choice | Approach |
|---|---|---|
| Best | Lowest level that clears the goal | Single-turn tool → multi-turn → supervised goal |
| 2nd | Supervised goal agent with write gates | Level 3 with approvals |
| 3rd | Long-running / exception-only | Only after eval maturity |
Wrong choice: Jump to unattended autonomy to match competitor marketing.
Why best is best: Extra freedom often reduces reliability until evals and policy catch up.
Decision 8: Is latency and cost compatible with multi-turn tool use?
Scenario: The user experience needs a sub-second answer on a mobile screen.
| Rank | Choice | Approach |
|---|---|---|
| Best | Single completion or one tool call | Cache, classify, retrieve once |
| 2nd | Async agent job with progress UI | User is not blocked on every turn |
| 3rd | Synchronous multi-turn agent | Only when wait is acceptable |
Wrong choice: A 12-tool loop behind a "instant answer" button.
Why best is best: Agents multiply tokens and wall time; product constraints must win.
Decision 9: Who owns escalation when the agent fails?
Scenario: The loop hits max turns or low confidence.
| Rank | Choice | Approach |
|---|---|---|
| Best | Named human queue or on-call path | Ticket with trace link |
| 2nd | Safe degrade to draft / no-op | User completes manually |
| 3rd | Retry blindly | Only with strict budgets |
Wrong choice: Silent failure or endless retries with no owner.
Why best is best: Production agents are ops systems; ownership is part of fit.
Decision 10: Do you have a golden task set before scaling users?
Scenario: A shiny demo on three cherry-picked prompts.
| Rank | Choice | Approach |
|---|---|---|
| Best | ≥20-50 golden tasks with expected outcomes | Track success, cost, interventions |
| 2nd | Small golden set + weekly expansion | Explicit coverage gaps |
| 3rd | Demo-only validation | Spike, not launch |
Wrong choice: Scale traffic on vibes and executive screenshots.
Why best is best: Fit is empirical. Without tasks, "good fit" is a slogan.
Decision 11: Is a simpler non-agent product the honest answer?
Scenario: After Decisions 1-10, several "Best" rows pointed away from agents.
| Rank | Choice | Approach |
|---|---|---|
| Best | Ship the simpler product | Script, workflow, RAG assistant, copilot |
| 2nd | Narrow agent slice beside the simple path | Route only hard cases to the loop |
| 3rd | Full agent rewrite of a working system | Rare; needs clear ROI |
Wrong choice: Rebuild a reliable workflow as an agent for branding.
Why best is best: The framework's job is to protect value, not to force the word "agent."
Applying the Decisions in Order
| Stage | Decisions | Exit criterion |
|---|---|---|
| Problem truth | 1-2 | Verifiable goal; path known vs branching |
| Feasibility | 3-5 | Tools, risk gates, traces |
| Design | 6-8 | Shape, autonomy, latency/cost |
| Launch | 9-11 | Owner, evals, honest non-agent option |
If any mandatory Best path is blocked, default to NO-GO as an agent and pick the 2nd-best product shape instead.
FAQs
Do all eleven decisions need a "Best" outcome?
Decisions 1, 3, 4, 5, 9, and 10 are near-mandatory for production autonomy. Others can accept 2nd place with documented risk.
How long should a fit review take?
A serious first pass fits in 60-90 minutes with eng + product + an owner for risk. Deep spikes follow only on GO or GO-with-caveats.
What if leadership already announced an agent?
Still run the framework. Ship the highest-ranked feasible choice (often a gated copilot) and keep the public name if you must, but do not expand write tools to match the press release.
Can I skip tool readiness if the model is frontier-class?
No. Model quality does not create APIs, auth, or dry-run modes.
Where do multi-agent designs show up in this framework?
After Decisions 6-7. Multi-agent is an architecture choice, not a substitute for fit, tools, or evals.
How does this differ from a model evaluation rubric?
Model evals score capability. This framework scores whether an agent system should exist for the problem at all.
What artifact should we store?
A one-page brief: goal, decisions 1-11 picks, tool list, autonomy level, eval set size, escalation owner.
Is "copilot" a failure outcome?
No. Copilot is often the correct product when judgment is subjective or stakes are high.
How often should we re-run the framework?
After major tool changes, incidents, or quarterly portfolio reviews. Autonomy should be allowed to decrease.
What is the most common false GO?
Path is actually known (should be a workflow), but the team wants agent branding and skips evals.
How do premature-use-case signals relate?
They are the red-flag cheatsheet version of failed decisions here (especially 3, 4, 8, 10).
Can startups with tiny teams pass Decision 9?
Yes if one named person owns escalations and the agent fails closed. "Nobody" is not an owner.
Related
- Use Cases for AI Agents Basics - practice mini go/no-go scoring
- Task Shape Matters More Than Industry When Picking a Use Case - Decision 6 depth
- Signals a Use Case Is Premature for an Agent Today - red-flag companion
- Where Agents Outperform Traditional Automation (and Where They Don't) - Decision 2 context
- Scope Rules: When a Problem Actually Needs an Agent - fundamentals rules view
Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana
@solana/web3.js+@solana/spl-token.