Security Checklist Before Shipping an Agent to Production
Use this as a go/no-go gate before real users, real data, or real money touch an agent with tools.
Tick items only when evidence exists in code, config, or a linked doc - not when they are "planned for next sprint."
How to Use This Checklist
- Walk sections A → F in order. Earlier failures make later polish irrelevant.
- Assign an owner per section (engineering, security, platform).
- Attach artifacts: tool allowlist, network policy, secret inventory, kill-switch drill.
- Re-run after any new tool, model path, tenant type, or autonomy increase.
- Pair with Agent Security Best Practices for ongoing habits and with prompt-injection work for content-level attacks.
A - Scope, Threat Model, and Autonomy
- 1. Named use case and user population - who runs the agent, on which workflow.
- 2. Explicit non-goals - shell on prod hosts, arbitrary HTTP, unsupervised payouts, etc. listed where relevant.
- 3. Threat model written - injection via user content/tools, secret theft, data exfil, abuse of spend, lateral movement.
- 4. Autonomy level recorded - draft / recommend / supervised execute / narrow auto, with criteria.
- 5. Success and kill criteria - what metrics pause or disable the agent.
- 6. Data classes in scope - public / internal / PII / secrets / regulated, labeled for prompts and tools.
B - Tools and Least Privilege
- 7. Tool allowlist per role - no kitchen-sink registry in production.
- 8. Each tool has a single clear purpose - intent-shaped APIs, not generic supertools.
- 9. Schema validation on all tool args - types, ranges, enums, length caps.
- 10. Path/workspace jail for file tools - no host home, no secret mounts.
- 11. No blanket shell - raw
run_shellabsent from prod registry (or break-glass only with approval). - 12. Irreversible actions inventory - money, delete, external send, prod change - listed with gates.
- 13. Human approval enforced in host code for high-risk actions, not only in the prompt.
- 14. Rate and amount caps - refunds, emails, rows, API calls bounded per run and per tenant.
C - Sandboxing and Execution Isolation
- 15. Untrusted code runs outside the API process - container, microVM, or equivalent.
- 16. Code sandbox network default deny -
--network noneor equivalent unless justified. - 17. Read-only root + limited writable tmp for sandboxes.
- 18. Non-root user, dropped capabilities, no new privileges.
- 19. CPU / memory / PID / time limits with hard kill.
- 20. Output truncation before observations re-enter the model.
- 21. Negative tests in CI - import/network/file escape attempts fail closed.
- 22. No Docker socket or privileged mounts exposed to agent code.
D - Credentials and Secrets
- 23. Secret inventory complete - every key, token, and service account listed with owner.
- 24. No master/admin keys in the agent - scoped roles only.
- 25. Short-lived or rotatable credentials - TTL or rotation schedule documented.
- 26. Secrets never in prompts, few-shots, or model-visible tool args.
- 27. Sandbox env scrubbed - code-exec cannot read provider or DB credentials.
- 28. Per-environment keys - dev/stage/prod separated.
- 29. Spend limits on model keys - provider/key caps configured (verify at build).
- 30. Revocation path tested - can disable a key without full outage theater.
- 31. Multi-tenant isolation - no shared downstream token across customers.
E - Network Egress and Data Boundaries
- 32. Per-tool destination allowlist - named hosts, not open internet.
- 33. Metadata and link-local ranges blocked - including cloud metadata IPs.
- 34. Private network access only via named internal services - not broad RFC1918.
- 35. Redirects re-validated - SSRF-safe HTTP client behavior.
- 36. Model provider egress separated from untrusted sandbox egress.
- 37. DLP/redaction on logs and traces - secrets and raw PII stripped or access-controlled.
- 38. Tenant data isolation - indexes, caches, memory stores cannot cross tenants.
F - Runtime Controls, Audit, and Incident Readiness
- 39. Max turns / timeouts / cost budgets enforced in the host for every loop.
- 40. Kill switch - disable tools or agent globally without redeploying the whole product (drill done).
- 41. Run reconstruction - run_id, tools, policy decisions, approver, versions.
- 42. Policy versioning - allowlists and prompt/policy packs pinned per run.
- 43. Alerting - deny spikes, spend spikes, error storms, unusual destinations.
- 44. On-call runbook - secret leak, runaway loop, bad autonomous action.
- 45. Rollback - previous tool pack / prompt / model config restorable quickly.
- 46. Security review sign-off - named reviewer for first launch or major expansion.
- 47. Dependency and image patch plan - base images and tool servers on a cadence.
- 48. Untrusted content policy - tool outputs and user docs treated as data, not instructions (link to injection controls).
Evidence Pack (Attach Before Go-Live)
| Artifact | Linked? | Owner |
|---|---|---|
| Threat model one-pager | ||
| Tool allowlist + approval matrix | ||
| Sandbox config (network, limits, image digest) | ||
| Secret inventory + rotation/revoke notes | ||
| Egress allowlist + deny test results | ||
| Kill-switch drill notes | ||
| Sample redacted audit trace | ||
| Incident runbook link |
Minimum Gate by Autonomy Level
| Autonomy | Minimum beyond "docs exist" |
|---|---|
| Draft only | B7-B9, D26, E37, F39, F41 |
| Recommend | + B12-B14, D23-D25, F43 |
| Supervised execute | + C15-C21, D27-D31, E32-E36, F40, F44 |
| Narrow unsupervised | + full E, stronger B13 dual control where needed, F46 formal sign-off |
If you cannot staff kill switch and on-call, do not enable unsupervised execute.
FAQs
Is this checklist only for customer-facing agents?
No. Internal agents with prod credentials often have higher blast radius because people trust them more.
What is the single most common ship blocker?
Generic shell or open HTTP tools still enabled "temporarily," plus long-lived admin API keys in the worker env.
Can we launch without containers if we have no code-exec tool?
Yes for isolation of code, but you still need least-privilege tools, scoped creds, egress policy, and budgets.
How does this relate to SOC2 or pen tests?
This is an agent-specific control list that feeds those programs. Map items into your existing control library.
Do we need every box for a private beta?
You need every box that matches your data and autonomy. Smaller audience does not shrink shell risk if keys are real.
Who owns the final go/no-go?
Joint: product owner + engineering + security. Security should be able to say no on B/C/D/E failures.
How often do we re-run after launch?
On every tool add, privilege increase, new tenant type, major model/provider change, and on a fixed calendar review.
Where should injection red-teaming sit?
Before unsupervised tools touch untrusted content. Item 48 is the checklist hook; use the prompt-injection section for methods.
What if leadership wants speed over gates?
Ship draft-only or supervised modes first. Document residual risk if anyone insists on unsupervised execute without controls.
Are framework defaults enough to tick boxes?
No. LangGraph, CrewAI, OpenAI Agents SDK, and others do not replace your allowlists, sandboxes, and secret layout.
How do we prove egress deny works?
Automated tests plus a staging probe against metadata IP, random public host, and an allowlisted host.
What belongs in the runbook vs the checklist?
Checklist is go-live evidence. Runbook is what humans do when alerts fire after launch.
Related
- The Principle of Least Privilege Applied to Agent Tools
- Agent Security Basics
- Containerized Sandboxes for Code-Executing Agent Tools
- Scoping API Keys and Credentials Passed to an Agent
- Never Grant Blanket Shell Access in Production: What to Do Instead
- Network Egress Controls for Agents That Call External Tools
- Agent Security Best Practices
- How Prompt Injection Attacks Actually Work
- What Agent Observability Actually Needs to Capture
Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana
@solana/web3.js+@solana/spl-token.