Team Rules for Reviewing and Approving Agent Changes
These rules make agent behavior changes reviewable, reversible, and owned.
Prompts, tool schemas, model maps, and retrieval configs change production behavior as much as application code - often with smaller, less obvious diffs.
Without process rules, teams ship silent quality cliffs, privilege expansions, and cost spikes under the label "quick config tweak."
How to Use This List
- Apply these rules in PR templates and CODEOWNERS (or equivalent), not only in a wiki.
- Require evidence links (eval run, trace sample, threat note) before merge on behavior changes.
- Scale rigor with blast radius: read-only copy tweaks are lighter than new write tools.
- Revisit owners and checklists when a new team starts shipping agents on the shared platform.
Classification Rules (What Kind of Change Is This?)
- Classify every agent PR by blast radius. Copy-only, model-map, tool schema, new tool, autonomy increase, or security-sensitive.
- Treat prompts, model maps, and tool schemas as production artifacts. They version with code and roll back with code.
- Separate experiment branches from production defaults. Feature flags or config layers beat editing the only path live.
- Name the user-visible behavior change in the PR description. If you cannot state it, you are not ready to review it.
- Link related incidents or tickets when the change is a fix. Retros without code ownership recreate the same bug.
Reviewer and Ownership Rules
- Assign platform or tech-lead CODEOWNERS for agent defaults. Stops, budgets, model policy, and security baselines need owners.
- Require a second reviewer for tool privilege or autonomy increases. Self-merge is forbidden for blast-radius expansion.
- Require security review when adding shell, broad HTTP, code-exec, or new credential scopes.
- Require product or domain review when success criteria or user-facing policy changes.
- Keep a small on-call or owner list for each production agent. "Everyone owns it" means no one does.
Evidence Rules (What Must Accompany the PR?)
- Attach eval results for behavior-changing PRs. Golden-suite pass/fail or graded summary, not only a screenshot of one chat.
- Include before/after traces for non-trivial loop or tool changes. Show a representative success and a failure path when relevant.
- Document cost impact when model tier or turn budget changes. Direction and order of magnitude beat silence.
- List new side effects and the gate that controls them. Approval matrix or host check reference.
- Link the threat-model note for new tools that touch money, PII, or external send.
- Refuse "works in playground" as the sole merge rationale for production agents.
Merge and Rollout Rules
- Prefer canary or flag-gated rollout for high-traffic agents. Full cutover without a kill path is a process failure.
- Ship prompt, tool, and model-map versions that must move together as one release unit when they are coupled.
- Require a rollback plan in the PR for material behavior changes. Know which versions restore last known good.
- Block merge if CI eval gates are red, unless an explicit, time-boxed waiver with owner is recorded.
- Do not disable stop conditions, budgets, or authz checks to green a demo. Fix the agent or shrink scope instead.
- Record exceptions with owner, reason, and expiry date. Expired exceptions reopen as debt.
Prompt and Config Diff Rules
- Prefer readable, minimal prompt diffs. Large unreviewed rewrites hide behavior landmines.
- Call out deleted constraints. Removing a "never" line is a security or product change, not cleanup.
- Avoid embedding secrets, env-specific hosts, or personal accounts in prompts.
- Keep few-shot examples free of real customer data unless the dataset is approved and redacted.
- Review tool description changes as carefully as handler code. Descriptions steer model tool choice.
Cadence and Culture Rules
- Run a periodic (for example quarterly) review of agent defaults and exceptions. Drift accumulates quietly.
- Turn each incident class into a checklist item or rule update. One-off patches are not enough.
- Share postmortems that name missing process controls, not only missing model intelligence.
- Onboard new agent authors with this list before first production PR.
- Keep the rule set short enough to enforce. A manifesto nobody checks is not a control.
PR Review Cheatsheet
| Change type | Minimum reviewers | Minimum evidence |
|---|---|---|
| Copy-only prompt polish | 1 peer | Spot-check tasks |
| Model tier / map change | 1 peer + platform aware | Eval suite + cost note |
| Tool schema additive | 1 peer | Handler tests + eval smoke |
| New write tool or broader scope | 2 reviewers | Tests, evals, threat note, gates |
| Autonomy increase | 2 + tech lead | Eval, rollout plan, kill switch |
| Shell / code-exec / egress widen | Security + eng | Isolation design + negative tests |
FAQs
Why not treat prompts as ordinary text files?
They change runtime behavior under non-determinism and tool use.
Review them with the same seriousness as business logic that moves money or data.
How big should an agent PR be?
As small as possible while remaining coherent.
Split "new tool" from "rewrite system prompt" when you can.
What if evals are flaky?
Fix flakiness or use statistical thresholds; do not delete the gate.
An ignored red gate trains the team to ignore all gates.
Do hotfixes get a free pass?
Hotfixes get a faster path with a mandatory follow-up PR, owner, and retro - not a permanent waiver of evidence.
Who can approve an exception to these rules?
The owner of the agent platform standards (tech lead or equivalent), in writing, with expiry.
How do multi-repo setups handle ownership?
CODEOWNERS (or review rules) per repo plus a documented platform owner for shared defaults.
Should product managers review prompt changes?
When user-facing policy, tone SLAs, or escalation rules change - yes.
Pure technical stop-condition edits may stay eng-owned.
What about personal or prototype agents?
Prototypes can be looser if they cannot reach production credentials or customers.
Promotion to production adopts the full review bar.
How do we prevent rubber-stamp reviews?
Require checklist answers and evidence links in the PR template; rotate reviewers; measure post-merge incidents by skipped items.
Is pair-prompting a substitute for review?
No.
Collaboration helps authoring; independent review catches missing gates and privilege creep.
How should we version prompts in Git?
Plain files or structured config with clear paths, history, and owners - avoid unreviewed edits only in a hosted UI without export.
What is the first process rule to implement?
Classify blast radius + forbid self-merge on tool/autonomy increases + require eval evidence on behavior changes.
Related
- How Opinionated Rules Prevent Agent Production Incidents - why process rules prevent drift.
- The Core Agent Rules: A Quick-Reference List - condensed review rows.
- Observability and Eval Rules for Production Agent Systems - evidence standards for PRs.
- Security Guardrail Rules Every Tech Lead Should Enforce - security bar reviewers enforce.
- AI Agent Rules Best Practices - checklist form including governance items.
- Versioning Prompts and Tool Schemas Alongside Code - artifact versioning.
- Feature Flagging Agent Behavior Changes - safe rollout.
- Regression Testing an Agent After a Model or Prompt Change - regression evidence.
Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana
@solana/web3.js+@solana/spl-token.