AI engineering interview questions by topic
Every topic real AI-era interviews test — from RAG and LLM evaluation to prompt injection, agents, and MLOps. Pick a topic to practice reported questions, scored on a production rubric against a golden answer.
- RAG ArchitectureDesign retrieval-augmented generation systems end to end — chunking, embeddings, hybrid retrieval, reranking, grounding, and diagnosing whether a failure is retrieval or generation.
- LLM EvaluationProve an LLM system works: eval design, LLM-as-a-judge pipelines, faithfulness and relevance metrics, regression tests, and measuring quality on real production data.
- Agent OrchestrationBuild multi-step agents: tool and function calling, planning and control loops, state, retries, and keeping long agent workflows reliable and debuggable.
- Token EfficiencyMake LLM systems economical — prompt and context sizing, caching, batching, and the token/latency/cost trade-offs interviewers expect you to reason about.
- Stakeholder CommunicationThe forward-deployed half of the job: scoping ambiguous problems with a customer, translating requirements, and communicating trade-offs to non-engineers under pressure.
- Vector DatabasesEmbeddings and vector search in production: index choice, similarity vs hybrid search, metadata filtering, freshness, and scaling retrieval to millions of documents.
- Prompt EngineeringProduction prompting: system prompt design, structured output, few-shot patterns, context-window management, and making prompts robust rather than brittle.
- RAG & RetrievalRetrieval quality for RAG — chunking strategy, reranking, multi-hop retrieval, and knowing when a deployment actually needs retrieval versus a bigger context window.
- MLOpsShip and operate ML: training pipelines, experiment tracking, reproducibility, CI/CD for models, and the deployment glue that turns a notebook into a service.
- LLM APIs & IntegrationWorking with model APIs: streaming, function calling, rate limits, retries and backoff, and integrating an LLM cleanly into a real application.
- Code ComprehensionUnderstand a codebase you did not write: tracing control flow, spotting the bug, and explaining what the system does before you change it.
- AI-Assisted DebuggingDebug with an AI assistant the way you would on the job — prompting the model effectively, verifying its suggestions, and catching where it leads you wrong.
- Bug DiagnosisRead unfamiliar code and find the defect: reasoning about failure modes, forming hypotheses, and isolating the root cause the way a real debugging round expects.
- SQLAdvanced SQL for data and AI roles: window functions, joins, aggregation, and writing correct, efficient queries against real schemas.
- Prompt InjectionThe defining AI-security topic: direct and indirect prompt injection, jailbreak patterns, and layered defenses that hold up when the input is adversarial.
- Agent DesignArchitect agentic systems: tool interfaces, guardrails, memory, cost and latency budgets, and designing for failure in multi-agent workflows.
- Model TrainingTraining and optimization: loss functions, regularization, hyperparameter tuning, and diagnosing why a model is not learning.
- Red TeamingAttack AI systems to make them safer: adversarial testing, jailbreak discovery, and probing for the failure modes a threat model missed.
- Model ServingServe models in production: packaging, inference APIs, batch vs real-time, autoscaling, and the latency/throughput trade-offs that matter under load.
- Data ModelingDimensional modeling and schema design: fact/dimension tables, normalization trade-offs, and modeling data so it stays queryable as it grows.
- Output ValidationConstrain and validate model outputs before they hit downstream systems — schema enforcement, sanitization, and safe handling of untrusted generations.
- Fine-TuningAdapt foundation models: when to fine-tune vs prompt vs RAG, SFT and PEFT, data curation, and evaluating whether the tune actually helped.
- Monitoring & DriftKeep production models healthy: quality and data drift, alerting, retraining triggers, and monitoring an LLM system you cannot unit-test.
- Evaluation & MetricsChoose the right metric and validation strategy, run error analysis, and prove a model or system is actually better — not just different.
- Cost & LatencyDesign to a latency budget and a cost ceiling: caching, payload sizing, model routing, and the trade-offs behind a fast, affordable deployment.
- PII HandlingProtect sensitive data: PII detection and redaction, data minimization, and preventing leakage through model inputs, outputs, and logs.
- Reliability & IdempotencyBuild systems that survive retries and duplicates: idempotent writes, ordering, late data, and recovering cleanly from partial failure.
- dbt & TransformationThe transformation layer: dbt models, tests, incremental builds, and keeping a warehouse transformation pipeline reliable and documented.
- Guardrail ArchitectureLayered runtime safety: input/output guardrails, content policies, and designing defenses that degrade gracefully instead of failing open.
- Access & GovernanceControl who can do what: RBAC, tenant isolation, least privilege, and field-level access in AI systems that touch sensitive data.
- SnowflakeWarehouse engineering on Snowflake: performance, cost, storage and compute separation, and modeling data for scale.
- Guardrails & SafetySafety filters and content policies as a system: what to block, how to layer defenses, and how to keep guardrails from breaking legitimate use.
- RefactoringImprove code without changing behavior: identifying smells, safe incremental refactors, and keeping tests green as you go.
- System Design & ScaleDesign services that scale: capacity estimation, data flow, bottlenecks, and the architecture trade-offs a system-design round is really testing.
- AI SecurityThe AI-security surface end to end: threat modeling, prompt injection, output validation, and reasoning about risk in LLM-powered systems.
- Adversarial MLAttacks on models themselves: evasion, data poisoning, model extraction, and building robustness against adversarial inputs.
- EU AI Act & ComplianceAI governance in practice: risk classification, documentation and audit trails, and designing systems that can prove their controls.
- Drift DetectionCatch data and concept drift before it degrades production, and design the monitoring and retraining triggers that respond to it.
- SparkDistributed data processing with Spark: partitioning, shuffles, joins at scale, and writing transforms that do not fall over on big data.
- Jailbreak DefenseDefend against jailbreaks: recognizing attack patterns, layering mitigations, and testing that your defenses actually hold.
- API DesignDesign clean, evolvable APIs: resource modeling, versioning, pagination, error semantics, and contracts that survive change.
- Streaming DataReal-time pipelines: event-time processing, exactly-once semantics, windowing, and keeping a streaming system correct under load.
- QuantizationShrink models for cheaper, faster inference — quantization schemes, accuracy trade-offs, and when the speedup is worth the quality cost.
- AirflowOrchestrate pipelines with Airflow: DAG design, scheduling, dependencies, backfills, and building retryable, observable workflows.
- Feature StoresManage features for ML: online/offline consistency, point-in-time correctness, and serving features reliably to training and inference.
- Distributed TrainingTrain at scale: data and model parallelism, synchronization, and the infrastructure trade-offs of multi-GPU and multi-node training.
- Prompt Engineering for CodePrompting models to write and fix code: framing tasks, constraining output, and verifying generated code before you trust it.