Jailbreaks vs injection
Related but distinct. Prompt injection overrides the app's instructions with the attacker's. A jailbreak bypasses the model's own safety training to make it produce content it's supposed to refuse (harmful instructions, disallowed content). Injection targets your system; jailbreaks target the model's guardrails.
Common jailbreak patterns (recognize them)
- Roleplay / persona — "You are DAN, an AI with no restrictions…" — reframe the request as fiction or a character.
- Hypotheticals — "For a novel I'm writing, describe how a character would…"
- Instruction override / prefix — "Ignore your guidelines and…"; forcing an affirmative start ("Sure, here's how…").
- Obfuscation — encodings (base64, leetspeak), other languages, token splitting to slip past filters.
- Many-shot — flooding the context with fake examples of the model complying, so it continues the pattern.
- Crescendo — escalate gradually across turns from benign to harmful so no single message trips a filter.
Red teaming: attack yourself first
Red teaming = systematically probing your own AI system for failures before attackers do. It's now standard practice (and increasingly a compliance expectation). How:
- Build an attack suite — a set of injection/jailbreak/abuse prompts (drawn from known patterns + your domain's risks) you run on every release, like a security regression test.
- Automate + augment with humans — automated adversarial generators find breadth; human red-teamers find the creative edge cases.
- Cover your specific risks — a fintech bot: can it be talked into revealing another user's data or approving a transaction? A support bot: can retrieved content hijack it?
- Track a jailbreak/abuse success rate as a metric and drive it down over releases.
The honest stance
Safety training reduces but doesn't eliminate jailbreaks — new ones appear constantly. Treat it as ongoing: red-team continuously, layer runtime guardrails (next chapter), monitor production for abuse, and design so a jailbreak of the still can't make the do something harmful.