Code Review Agents: Automated PR Feedback at Scale
Code review agents read a diff (and often surrounding context), check it against policies and risk patterns, and leave structured feedback on the pull request.
//
Search across all documentation pages
Code review agents read a diff (and often surrounding context), check it against policies and risk patterns, and leave structured feedback on the pull request.
They scale first-pass review. They do not own merge accountability.
| Capability | What it looks at | Typical output |
|---|---|---|
| Diff comprehension | Changed hunks + nearby symbols | Summary of intent vs risk |
| Defect heuristics | Nulls, races, authz gaps, error swallowing | Line comments with rationale |
| Style / convention | Team rules, linters, naming | Nits separated from blockers |
| Security patterns | Secrets, injection sinks, unsafe APIs | High-severity findings first |
| Test adequacy | Diff vs test files | "Missing coverage on X" notes |
| Policy checks | OWNERS, license, migration rules | Pass/fail checklist items |
| Input | Why it matters | Minimum viable |
|---|---|---|
| Unified diff / PR files API | Core change set | Required |
| Base branch SHA | Correct merge base | Required |
| CI status / logs | Ground feedback in real failures | Strongly recommended |
| Repo conventions file | Encode "how we review here" | README / CONTRIBUTING / rules file |
| Historical PR comments | Reduce repeated nits | Optional |
| Severity taxonomy | Blocker vs nit language | Required for signal quality |
| Mode | Behavior | Use when… |
|---|---|---|
| Advisory comments | Posts notes; never blocks | First rollout |
| Checklist report | Single summary comment with scores | Noisy repos; prefer one thread |
| Blocking status check | Fails CI on severity ≥ threshold | Mature rules; low false positives |
| Auto-request changes | Uses review API "request changes" | Only with tight, tested policies |
| Silent shadow | Logs findings without posting | Calibration week |
| Severity | Examples | Human action |
|---|---|---|
| S0 Blocker | Auth bypass, secret commit, data loss risk | Must resolve before merge |
| S1 High | Likely prod bug, missing migration, unsafe default | Resolve or explicit waiver |
| S2 Medium | Edge-case bug, weak tests on critical path | Fix or track follow-up |
| S3 Low / nit | Style, naming, comment typos | Optional; limit volume |
| Info | Architecture suggestion | Never block |
| Excels | Struggles |
|---|---|
| Mechanical policy (license headers, forbidden APIs) | Product intent and UX taste |
| Common bug classes in popular languages | Deep domain invariants with no tests |
| Consistent nit reduction for juniors | Cross-service design trade-offs |
| Secret and dependency red flags | Performance judgment without profiles |
| "Tests didn't touch changed code" | Flaky CI interpretation |
| Stage | Agent role | Tip |
|---|---|---|
| Pre-commit / IDE | Local quick checks | Keep under a few seconds |
| PR open | First-pass summary | Post once; update on push |
| CI parallel | Security + test-gap scan | Cache repo context |
| Before merge | Re-check latest commit only | Avoid commenting on outdated lines |
| Post-merge | Rare; prefer pre-merge | Use for audit sampling only |
| Anti-pattern | Why it hurts | Prefer |
|---|---|---|
| Wall of S3 nits | Authors ignore everything | Cap nits; separate severity |
| Blocking on style the linter already covers | Duplicate noise | Let linters own style |
| No waiver path | Emergency merges stall | Explicit override + audit log |
| Reviewing generated files | Noise and token waste | Path denylist |
| Hidden model prompts as "policy" | Unreviewable rules | Versioned rules in-repo |
| Auto-merge on agent LGTM alone | Accountability hole | Human for risky paths |
| Metric | Healthy signal | Warning signal |
|---|---|---|
| Precision of S0/S1 | Authors agree findings are real | Constant "false alarm" replies |
| Time-to-first-feedback | Minutes after PR open | Hours later than human review |
| Comment volume per PR | Bounded, severity-sorted | Dozens of undifferentiated nits |
| Human override rate | Occasional, explained | Always ignored or always rubber-stamped |
| Escaped defects in agent-reviewed PRs | Flat or down | Up after bot adoption |
| Section | Content |
|---|---|
| Always flag | Secrets, eval, raw SQL concat, disabled auth |
| Never flag | Pure formatting already in CI, vendored code |
| Language packs | Python/TS/Go-specific footguns you care about |
| Path policies | payments/** always human-reviewed |
| Tone | "Suggest" vs "Require"; no shaming language |
| Output format | Severity, file:line, why, suggested fix |
No. They scale first-pass inspection. Humans still own design judgment, merge decisions, and accountability.
Usually no. Run advisory or shadow mode until S0/S1 precision is trustworthy.
Enough surrounding code to understand call sites and tests, plus project rules. Not the entire monorepo every time.
Severity caps, nit budgets, update-in-place summaries, and denylists for generated paths.
Those catch mechanical issues. Agents help with cross-hunk reasoning and policy narrative - overlapping but not identical.
In-repo, versioned, reviewable files. Prompt-only policy is invisible and drifts.
Yes with language-specific rule packs and path routing. One generic prompt over everything performs worse.
Treat agent-authored PRs the same or stricter. Require human review on sensitive paths regardless of who wrote the diff.
Always high severity. Prefer dedicated secret scanners in CI as a hard gate; the agent is a backup narrative layer.
Yes for clear mechanical fixes. Keep patches small and never auto-apply on protected branches without policy.
Label a set of historical PRs with known issues; measure precision/recall on S0/S1; measure author satisfaction for two sprints.
High-volume low-value nits that train authors to ignore the bot.
Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana
@solana/web3.js+@solana/spl-token.
Reviewed by Chris St. John·Last updated Jul 16, 2026