The ML lifecycle is the full loop from defining a problem to keeping a model healthy in production. The single most important thing to internalize: it's a loop, not a line, and most ML-engineering work happens after the model is trained — in data, serving, monitoring, and retraining. Training is the part tutorials obsess over and the part you'll spend the least time on in a mature system.
Why this appears in interviews
The lifecycle is the scaffolding for every "design an ML system" answer. Interviewers use it to check whether you think end-to-end or fixate on modeling. A candidate who walks a problem through framing → data → features → training → serving → monitoring → retraining sounds like someone who has operated systems; a candidate who jumps to "I'd train a neural net" does not. It's also a diagnostic: when a production model breaks, knowing the lifecycle tells you which phase to inspect.
The mental model — the iceberg
Picture an iceberg. Above the water: model selection, training, and tuning — the visible, glamorous part. Below the water, far larger: problem framing, data pipelines, feature engineering, serving infrastructure, monitoring, and retraining. In real organizations, the submerged mass is the job. Internalizing this reframes every design question away from "which model?" toward "which pipeline, which features, which serving path, which monitors?"
The phases
1. Problem framing. Translate a business goal into an ML problem: what exactly are you predicting, what's the label, what does a wrong prediction cost, and what online metric defines success? Skipping this is the top cause of failed ML projects — you build a technically-fine model that optimizes the wrong thing.
2. Data collection and preparation. Where does training data come from, how much, how fresh, how is it labeled? Typically 60–80% of project time. Most production failures trace back here (drift, skew, leakage), so treat data as a first-class system, not a preprocessing step.
3. Feature engineering. Transform raw data into model inputs, computed consistently for training and serving (feature store) and free of leakage (no future information). Feature quality sets the ceiling on model performance.
4. Model training and evaluation. Choose an approach, train, and evaluate on a held-out set against a baseline. Start simple; a gradient-boosted or even logistic-regression baseline both ships fast and tells you whether a complex model is worth it. In mature teams this is a small slice of total effort.
5. Deployment. Package the model (container), stand up a serving path (batch or real-time), integrate with the app, and validate in staging. Deploy safely — shadow, canary, A/B — with a registry-backed rollback.
6. Monitoring and maintenance. Track feature distributions, prediction distributions, latency, and the business KPI. This phase never ends — a deployed model decays as the world changes.
7. Retraining. When monitoring detects drift/decay (or on a schedule), retrain through the same pipeline, gate the candidate against the current production model, and promote only if it wins. Then monitor again — the loop closes.
The loop, drawn
frame → data → features → train → evaluate(vs baseline)
│
deploy (canary/A-B)
│
monitor ──drift/decay──┐
│ │
business value │
└── retrain ◀───────┘
The arrow back from monitoring to retraining is the whole point: production ML is a control loop that keeps a model aligned with a moving world, not a project that ends at launch.
Two ideas that separate strong answers
- Baseline first. If your model can't beat the simplest reasonable approach (a heuristic, a majority-class predictor, last-value), you don't have a model worth deploying. Establishing the baseline is step one of evaluation, not an afterthought.
- Design for retraining and rollback on day one. "How will this model be retrained and reverted?" should be part of the initial design, not bolted on after the first incident. Reproducibility (versioned data + code + config) is what makes both possible.
How the lifecycle guides debugging
When a live model's metric drops, the lifecycle is your triage map: is it a data problem (phase 2 — a broken upstream feed, schema change), a feature problem (phase 3 — training-serving skew, leakage), a drift problem (phase 6 — the world changed), or genuinely a model problem (phase 4)? Most incidents are data or feature problems, not model problems — which is why experienced ML engineers check the pipeline before retraining.
Common interview mistakes
Mistake 1: Jumping to model selection. Problem framing, data assessment, and a baseline come first; the model is the middle of the story, not the start.
Mistake 2: No retraining/monitoring story. A complete design always includes how the model is monitored and retrained — omitting it signals prototype-only experience.
Mistake 3: Ignoring the baseline. Proposing a complex model with nothing to compare against.
Mistake 4: Treating the lifecycle as linear. It's a loop; the value is in the feedback from production back into data and training.
Key vocabulary
- Problem framing — Translating a business goal into a prediction target, problem type, and success metric.
- Baseline — The simplest reasonable model defining the floor of acceptable performance.
- Data leakage — Future/target information sneaking into training features, inflating offline metrics.
- Retraining trigger — The condition (schedule, drift, or data volume) that starts a new training run.
- Reproducibility — Recreating a model from pinned data + code + config; the basis for retraining and rollback.
- The loop — Monitoring feeding drift/decay signals back into retraining; production ML as a control loop, not a project.