Revisiting Old Agent Architecture Decisions as the Field Moves
An accepted ADR is not a forever spell.
In agent systems, models, frameworks, gateways, and protocols move fast enough that scheduled and triggered reviews are part of the architecture.
This ranked-decision list tells you what to revisit first, how often, and when to supersede vs leave alone.
How to Use This List
- Run the calendar cadence even when nothing feels wrong.
- Use ranked triggers when an event fires mid-quarter.
- Prefer dual-run + eval over big-bang rewrites when superseding.
- Record outcomes:
still valid,update pins, orsuperseded by ADR-0XX.
Ranked Review Priority (what to check first)
| Rank | Decision class | Why it ages fastest | Typical action |
|---|---|---|---|
| 1 | Model pins and tiers | New releases and price shifts monthly | Re-eval; retarget roles |
| 2 | Provider / gateway path | Fees, outages, policy, BYOK | Confirm path still legal and cheap enough |
| 3 | Tool privilege surface | New MCP servers and write tools | Re-assert least privilege |
| 4 | Orchestration topology | Single vs multi-agent cost in prod metrics | Merge or split with evidence |
| 5 | Framework / runtime | Majors, checkpoint formats, lock-in pain | Spike migrate or stay |
| 6 | Memory and retention | Privacy law + product memory features | Tighten retention; migrate store |
| 7 | Eval gates and SLOs | Traffic shape changes | Recalibrate thresholds |
| 8 | HITL and kill switches | Autonomy creep | Re-map irreversible actions |
Always start at rank 1-2 on a quarterly review unless a higher-severity trigger says otherwise.
Decision 1: Did a new model generation ship?
| Rank | Choice | Approach |
|---|---|---|
| Best | Re-run gold set on candidate pins; swap roles only if metrics win | Keep roles stable; change pins |
| 2nd | Canary new pin on one surface | Limit blast radius |
| 3rd | Ignore until annual planning | Only if spend and quality are already excellent |
Wrong choice: Silent production pin edits with no eval. Why best is best: You capture upside without rewriting architecture stories.
Decision 2: Did cost per successful run jump?
| Rank | Choice | Approach |
|---|---|---|
| Best | Inspect turns, model mix, and thrash; then retier or bound | Fix host policy before blaming one vendor |
| 2nd | Swap mid-tier pin or add triage role | Document in model decision record |
| 3rd | Big-bang multi-agent rewrite for cost alone | Coordination may raise cost |
Wrong choice: Cutting all evals and logging to "save money." Why best is best: Agent cost is usually loop-shaped, not only $/token.
Decision 3: Did we have a provider or gateway SEV?
| Rank | Choice | Approach |
|---|---|---|
| Best | Exercise documented fallback; file ADR amendment if fallback failed | Incidents test the access-path decision |
| 2nd | Add hybrid backup path | Direct + gateway or second provider |
| 3rd | Abandon multi-model forever out of anger | Overfitting to one outage |
Wrong choice: Emergency hardcode with no decision record update. Why best is best: The ADR should get smarter after every SEV.
Decision 4: Is the framework fighting us?
| Rank | Choice | Approach |
|---|---|---|
| Best | List concrete escape-hatch pain; spike one alternative for a week | Evidence over vibes |
| 2nd | Stay; invest in platform wrappers | If pain is training, not capability |
| 3rd | Rewrite all agents in a weekend | Portable tools or bust |
Wrong choice: New framework every quarter. Why best is best: Framework ADRs should supersede only with spike + migration plan.
Decision 5: Are multi-agent handoffs failing or expensive?
| Rank | Choice | Approach |
|---|---|---|
| Best | Measure handoff failure and duplicate tools; simplify topology if roles are fake | Merge specialists with identical allowlists |
| 2nd | Tighten handoff schema and budgets | Keep multi if privilege differs |
| 3rd | Add more agents to "fix" coordination | Usually worsens cost |
Wrong choice: Prompt-only "please coordinate better." Why best is best: Topology is an architecture decision; fix it with contracts and metrics.
Decision 6: Did compliance or data policy change?
| Rank | Choice | Approach |
|---|---|---|
| Best | Treat as hard trigger; re-validate gateway/direct and memory stores immediately | Legal beats convenience |
| 2nd | Segment traffic (PII path vs non-PII path) | Document hybrid |
| 3rd | Hope existing DPAs cover new tools | Audit failure waiting to happen |
Wrong choice: Shipping a new MCP server that exfiltrates customer data without review. Why best is best: Access path and tool ADRs are compliance artifacts.
Calendar Cadence
| Cadence | Scope | Output |
|---|---|---|
| Monthly | Model pins, cost/success, error slices | Pin table updates or explicit "no change" |
| Quarterly | Full ADR index review (status + triggers) | still valid / supersede plan |
| Per major release | Framework or protocol upgrades | Spike notes + migration ADR |
| Per SEV | Related decisions only | Amendment within 5 business days |
| Per new write tool class | Privilege + HITL ADRs | Accept or reject before enable |
Put the quarterly review on the team calendar with an owner - not "when we have time."
Review Meeting Agenda (45 minutes)
- Index - list ADRs
acceptedolder than 90 days. - Triggers - any fired since last review?
- Metrics - success, p95, $/success, max-turn stops, handoff failures.
- Pins - model and gateway pins still match records?
- Decisions - for each candidate: keep / update pins / supersede.
- Actions - tickets with owners; no orphan "we should."
Supersede vs Amend vs Leave
| Situation | Action |
|---|---|
| Same decision, new model id | Amend pins + date; keep ADR id |
| Forces changed (legal, topology) | Supersede with new ADR |
| Minor wording | Edit with PR; no status circus |
| Decision still right, doc stale | Refresh consequences; status stays accepted |
| System retired | Deprecated with pointer to replacement |
Anti-Patterns
| Anti-pattern | Better |
|---|---|
| Eternal accepted ADRs never opened | Quarterly pass |
| Rewrite without supersession links | Bidirectional ADR links |
| Review only when angry after outage | Calendar + triggers |
| Measuring framework hype, not evals | Gold set first |
| Changing topology and provider in one ADR | Split decisions |
FAQs
How many ADRs should a small team maintain?
Enough to cover binding choices (often 5-15), not one per ticket. Archive deprecated systems.
What if nobody remembers why an old ADR exists?
That is a review finding. Either reconstruct from git and metrics or supersede with a clean decision.
Should product managers attend ADR reviews?
For autonomy, cost, and user-visible degrade paths, yes. Skip pure package-upgrade pins.
How do we avoid thrashing decisions every month?
Require eval deltas and cost impact above a threshold before superseding topology or framework.
Is a "no change" outcome worth recording?
Yes. A dated still valid note prevents re-litigating the same argument without new evidence.
Do open-source agents need this rigor?
If others depend on your defaults, yes. Hobby scripts can keep a shorter pin changelog.
Where should review notes live?
In the ADR file (append a Review log) or a linked minutes doc committed next to ADRs.
What metrics are mandatory for agent ADR reviews?
Task success, cost per success, p95 run latency, stop-reason mix, and (if multi-agent) handoff failure rate.
Can feature flags replace revisiting ADRs?
Flags implement rollout. Reviews still decide whether the target architecture remains correct.
How soon after a framework major should we review?
Within one sprint of the major becoming relevant to your version pin - do not wait for a production break.
What is the highest-ROI single habit?
Monthly pin and cost review with a frozen gold set. Most field movement shows up there first.
How does this relate to custom runtime exit criteria?
Runtime exit cheatsheets are trigger sources for framework ADRs - link them in the quarterly agenda.
Related
- Why Agent Systems Need Their Own ADR Discipline - why decisions must be revisitable
- Documenting Model and Provider Decisions for Future Migrations - pin records
- Architecture Decisions Basics - supersession sketch
- An ADR Template for Agent Framework Selection - framework template
- Trade-off Guide: Single Agent vs Multi-Agent Architecture - topology
- Trade-off Guide: Managed Gateway vs Direct Provider Integration - access path
- When a Custom Runtime Stops Being Worth Maintaining - exit custom
- Architecture Decisions Best Practices - operating checklist
Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana
@solana/web3.js+@solana/spl-token.