Least privilege, everywhere
The load-bearing governance control (echoing the AI Security course): the system — and any AI/agent within it — accesses only what the authenticated user is authorized for. Concretely:
- RBAC / ABAC — role- or attribute-based access to data and actions.
- Row- and field-level access — a user (or the model acting for them) sees only their permitted rows/fields. In RAG, this means filtering retrieval by the user's permissions, not just tenant.
- Authorize the user, not the model — a model-driven action executes only against the user's entitlements; never grant the model broader access than the user has.
- Least privilege on tools & secrets — narrowly-scoped tools, secrets in a manager, rotation.
Audit trails (you must be able to show what happened)
High-risk AI (EU AI Act) requires automatic logging, and every serious governance regime expects an audit trail: who did what, when, what the AI decided, what data it used, and whether a human reviewed it. Engineer it:
- Log decisions + inputs + model/version + who + when, immutably and access-controlled.
- Make it queryable — "show every AI decision affecting user X" and "why did the system decide this?" must be answerable.
- Balance with privacy — audit logs contain sensitive data; protect and retain them appropriately.
Human oversight & accountability
- Human-in-the-loop for high-stakes or irreversible decisions (a person can review, intervene, override) — an EU AI Act high-risk requirement and a security control.
- Clear ownership — someone is accountable for the system's behavior; "the AI did it" is not a defense.
- Explainability — for consequential decisions, be able to explain the basis (the audit trail + the retrieved context/rationale).
Putting it together
Access control decides what the system can touch; audit trails record what it did; human oversight catches what it shouldn't do alone. Together they're how you make an AI system accountable — and how you demonstrate compliance when asked.