Why LLM apps are a new security problem
Traditional security assumes code and data are separate. LLMs erase that line: instructions and data arrive in the same channel — natural language — and the model can't reliably tell "content to process" from "commands to obey." That single property is the root of most LLM vulnerabilities.
The OWASP LLM Top 10 (the field's shared map)
Know these categories by name:
- LLM01 Prompt Injection — untrusted input overrides the intended instructions (the #1 risk; next chapter).
- LLM02 Sensitive Information Disclosure — the model leaks PII, secrets, or system-prompt content.
- LLM03 Supply Chain — compromised models, datasets, or plugins.
- LLM04 Data & Model Poisoning — malicious training/fine-tuning data.
- LLM05 Improper Output Handling — trusting model output as safe (it renders as HTML → XSS, runs as SQL → injection).
- LLM06 Excessive Agency — an agent with too much permission does real damage.
- LLM07 System Prompt Leakage — secrets/logic embedded in the system prompt exposed.
- LLM08 Vector/Embedding Weaknesses — poisoning the RAG index, cross-tenant leakage.
- LLM09 Misinformation — confident hallucinations relied on for decisions.
- LLM10 Unbounded Consumption — cost/DoS via expensive generation.
The mental model: never trust, always verify
Treat everything the model sees (user input, retrieved documents, tool results, web pages) as untrusted, and everything the model produces as untrusted until validated. The model is a powerful but manipulable component in the middle — security lives in the system around it (authorization, validation, sandboxing, human-in-the-loop), not in hoping the model behaves.
Defense in depth
No single control is enough. Layer them: input validation → grounding/prompt hardening → least-privilege tools + user-scoped authorization → output validation → monitoring/rate limits → human approval for high-stakes actions. The rest of this course walks the layers.