Guardrail Libraries and Content-Filtering Layers for Agents
Use this cheatsheet when you pick layers for filtering unsafe inputs, tool results, and agent outputs - and when you map off-the-shelf libraries to those layers without treating any package as a complete security boundary.
Names, versions, and APIs move quickly - verify at build against current docs for each library and model provider.
How to Use This Cheatsheet
- Stack mechanical controls (schemas, allowlists, sandboxing) under linguistic filters (classifiers, moderation, injection detectors).
- Prefer host-owned gates over "the model promised to be good."
- Measure false positive rate on real tickets before blocking production traffic.
- Re-eval after model or tool changes; filters drift when the attack surface grows.
A - Defense Layers (Outside-In)
| Layer | What it filters | Typical mechanism | Injection relevance |
|---|---|---|---|
| 1. Edge / auth | Who may talk to the agent | Authn, rate limits, WAF | Reduces spam-scale probing |
| 2. Input moderation | User text policy | Provider moderation APIs, open classifiers | Blocks some jailbreaks, not all indirect paths |
| 3. Untrusted isolation | Tool/RAG free text | Envelopes, dual-path facts | Primary design control |
| 4. Injection / jailbreak detectors | Override-like language | Rules, ML judges, dedicated scanners | Signal for log + soft block |
| 5. Tool argument validation | Executable intent | Pydantic, Zod, JSON Schema | Stops many side effects |
| 6. Output moderation | User-visible text | Toxicity, PII, secret scanners | Limits leaky replies |
| 7. Egress / sandbox | What tools can touch | Network policy, containers | Caps blast radius |
| 8. Human gates | High-impact actions | Approvals, break-glass | Last line for irreversibles |
No single layer replaces the stack. Libraries mostly help layers 2, 4, and 6; you still own 3, 5, and 7.
B - Library and Service Map
| Option | Role | Fits agents when... | Watch-outs |
|---|---|---|---|
| OpenAI Moderation API (or compatible) | Input/output policy classes | You already call OpenAI-compatible endpoints | Categories ≠ injection detection |
| Provider safety / shield features (Anthropic, Google, etc.) | Built-in filters | Single-provider stacks | Bypass via tool channels; verify coverage |
| NVIDIA NeMo Guardrails | Dialog rails, topical, some jailbreak flows | Conversational bots with Colang-like policies | Heavier runtime; map carefully to tool loops |
| Guardrails AI | Validators around LLM I/O, many hub validators | Python apps wanting declarative checks | Validator quality varies; keep host schemas |
| Llama Guard / prompt-guard style models | Safety / injection classifiers | You can host or call a second model | Latency, false pos/neg, version drift |
| Microsoft Agent Framework / Azure content safety | Enterprise content filters | Azure-centric deployments | Region/config dependent |
| LangChain / LlamaIndex callbacks & guards | Framework hooks | Already on those stacks | Hooks are only as strong as your validators |
| OpenAI Agents SDK guardrails (tripwires) | Input/output checks around agents | SDK-native Python/TS agents | Verify current API at build |
| Presidio / custom PII detectors | PII in/out | Support and HR agents | Language coverage |
| Secret scanners (regex + entropy) | API keys in outputs | Any agent that sees credentials | High FP on random tokens |
| Homegrown Pydantic/Zod registry | Tool contracts | Every production agent | Not a "library product" but essential |
Prefer boring host validation over a large dependency tree when the only requirement is tool safety.
C - Where to Place Filters in the Agent Loop
User input ──► [input moderation] ──► policy + planner
│
▼
tool calls ──► [arg schema]
│
▼
tool / RAG result ──► [isolate / detect]
│
▼
planner continues
│
▼
final answer ──► [output filter] ──► user- Run cheap rules before expensive LLM judges.
- Do not feed full raw tool dumps into a privileged planner "so the filter can see them" without a dual-path design.
- Apply output filters after the model, before UI, email, or webhooks.
D - Decision Snapshot
| Need | Prefer | Avoid as sole control |
|---|---|---|
| Stop bad tool args | Pydantic/Zod + allowlists | Prompt wording |
| Soft-detect injection phrases | Heuristics + optional classifier | Blocking all imperatives |
| Policy categories (hate, self-harm) | Provider moderation | DIY from scratch without evals |
| Enterprise audit story | Documented layered controls | One vendor checkbox |
| Low latency chat | Rules + schema; async LLM judges | Multiple serial judge models |
| Code-exec agent | Sandbox + egress deny | Content filter only |
E - Minimal Host Skeleton (Library-Agnostic)
from pydantic import BaseModel, ValidationError
class SearchArgs(BaseModel):
query: str
limit: int = 5
def moderate_stub(text: str) -> bool:
# Swap for provider moderation or Llama Guard-style call
return "ignore previous instructions" not in text.lower()
def handle_user(text: str, proposed_tool: dict | None) -> str:
if not moderate_stub(text):
return "blocked: input policy"
if not proposed_tool:
return "ok: no tool"
if proposed_tool.get("name") != "search":
return "blocked: tool"
try:
SearchArgs.model_validate(proposed_tool.get("args") or {})
except ValidationError:
return "blocked: args"
return "ok: would search"Swap moderate_stub for a real service; keep the schema gate either way.
F - Evaluation Checklist for Any Guardrail Package
| Check | Why |
|---|---|
| Documented input vs output hooks | Agents need both plus tool-result hooks |
| Sync latency budget | Multi-judge stacks blow P95 |
| Offline eval set support | You must score FP/FN on your traffic |
| Failure mode fail-open vs fail-closed | Network blip must not disable gates silently |
| Logging / redaction | Injection attempts often contain sensitive text |
| License and data retention | Cloud filters may train or store |
| Escape hatches | Break-glass for support, not silent bypass in prod |
FAQs
Will a guardrail library stop prompt injection completely?
No.
Libraries reduce risk on language and some patterns. Isolation, least privilege, and validation remain mandatory for tool-using agents.
Should I put NeMo or Guardrails AI in front of every tool result?
Only if latency and FP rates are acceptable.
Many teams isolate/summarize tool results first, then run lighter detectors.
Is provider moderation enough for agents?
It helps on user chat policy.
Indirect injection in tools and executable tool abuse need host controls.
Open source classifier or hosted API?
Hosted APIs ship faster; self-hosted helps privacy and cost at volume.
Either way, keep mechanical gates local to your process.
How many layers are too many?
When P95 latency or false blocks exceed product tolerance.
Start with isolation + schema + one moderation path; add judges where incidents demand.
Do framework "guardrail" features replace this cheatsheet?
They implement hooks.
You still choose policies, schemas, and fail-closed behavior.
What should I log when a filter fires?
Trace id, layer name, rule id, redacted snippet, tool name, and user id - not raw secrets.
How often do I re-benchmark filters?
On model upgrades, new tools, and after any injection incident - plus a fixed quarterly pass.
Related
Related: How Prompt Injection Attacks Actually Work
Related: Output Validation with Pydantic and Zod Against Malicious Content
Related: Detecting and Logging Suspected Injection Attempts
Related: Prompt Injection & Guardrails Best Practices
Related: Security Guardrail Rules Every Tech Lead Should Enforce
Stack versions: Pins from the category manifest (verify at build): OpenRouter (~315+ models, July 2026 pricing/fees); LangGraph 1.0+; CrewAI 1.14+; Microsoft Agent Framework 1.0; Vercel AI SDK 6; Pydantic AI (latest); LlamaIndex (latest); OpenAI Agents SDK (latest + MCP); MCP (Linux Foundation governance); A2A (HTTP+SSE+JSON-RPC 2.0); Solana
@solana/web3.js+@solana/spl-token.