Guardrails = runtime controls around the model
Prompting and safety training aren't enough (Ch. 2–3), so wrap the model in runtime guardrails — programmatic checks on the way in and out:
Input guardrails (before the model):
- Detect and block/flag known injection & jailbreak patterns.
- PII detection — redact sensitive data before it reaches the model or your logs.
- Topic / scope checks — reject off-scope or clearly abusive requests.
- Rate limiting per user (cost + abuse).
Output guardrails (before you use the response):
- Validate structure — schema/format check; reject malformed output.
- Faithfulness / grounding — for RAG, verify claims are supported by context (block unsupported ones).
- Safety/content filters — block disallowed content, PII leakage, secrets.
- Sanitize for the sink — escape HTML (XSS), parameterize SQL, never
evalmodel output. This is OWASP LLM05.
Least privilege & authorization (the load-bearing control)
The guardrail that matters most for agents/tools: a model-influenced action executes only if the user is authorized for it — never on the model's say-so. Scope tools narrowly (read-only where possible), require human approval for irreversible/high-value actions, and sandbox tool execution. Even a fully jailbroken model can't exceed the user's real permissions.
Guardrail tooling
You can build guardrails yourself (a fast classifier or a small LLM "judge" as a safety pass) or use frameworks (Guardrails AI, NeMo Guardrails, Llama Guard / provider moderation endpoints). A common pattern: a cheap fast model screens input/output for safety before/after the main model.
Defense in depth, again
No single guardrail is complete. Layer input + output + authorization + monitoring, and assume each layer will sometimes fail. The goal isn't a perfect wall — it's making a breach need to defeat several independent controls to cause harm.