Feature Engineering, ML Engineer Stage 1

Feature engineering is the craft of transforming raw data into the inputs a model can actually learn from — and it remains, even in the deep-learning era, where most practical ML performance comes from. A great feature set with a simple model routinely beats a fancy model with raw inputs. It's also where the subtlest, most dangerous bugs live: leakage and training-serving skew. This is the highest-leverage skill in the track.

Why this appears in interviews

"What features would you use for X?" and "your model underperforms — what would you try?" are staples, and both are feature questions. Interviewers want to see that you can invent domain-grounded features, reason about encoding and leakage, and connect features to the production concerns (consistency, freshness) that make or break real systems. Strong feature reasoning signals hands-on ML experience more than knowing model architectures.

The mental model — raw ingredients vs a cooked dish

Raw data is customer_id, last_purchase_date, account_created, a free-text review, a timestamp. Features are what the model eats: days_since_last_purchase = 15, purchase_frequency_30d = 4.2, is_weekend = true, review_sentiment = 0.8. The model can only find patterns in the representation you give it — feature quality sets the ceiling on performance, and no amount of model tuning raises that ceiling.

Categories of features

Raw features — columns straight from the source. Easy, often low-signal.
Aggregations — statistics over a window: count/sum/avg/max of a user's activity in the last 7/30 days. High-signal, and the main source of leakage and skew bugs (the window must be identical offline and online).
Ratios and differences — "transaction amount ÷ user's average," "days since last login." Capture relative behavior, which is often more predictive than absolutes.
Interaction features — combinations, e.g. "weekend AND outside home city." Capture joint patterns a linear model can't infer alone.
Temporal features — hour-of-day, day-of-week, seasonality, recency; time is signal in almost every real system.
Embeddings — dense vector representations of high-cardinality categoricals (user IDs, items) or text, learned or pretrained.

Core transformation techniques

The mechanics you should be able to name and justify:

Encoding categoricals — one-hot for low cardinality; target/frequency encoding or embeddings for high cardinality (thousands of categories). One-hot on a million user IDs explodes dimensionality — a common mistake.
Scaling/normalization — standardize (z-score) or min-max for models sensitive to scale (linear models, neural nets, distance-based). Tree models don't need it. The scaler must be fit on training data only and shipped with the model (or the online store) to avoid skew.
Missing values — impute (mean/median/model-based), or add an explicit "is-missing" indicator when missingness is itself signal (a blank income field may correlate with fraud).
Outliers and skewed distributions — clip, or apply a log/Box-Cox transform so a few huge values don't dominate.
Binning/discretization — turn a continuous value into ranges when the relationship is non-linear and you want an interpretable, robust feature.
Text/other modalities — TF-IDF or embeddings for text; extraction for images/audio — feeding downstream features.

The two bugs that define production feature work

Data leakage — a feature that encodes information unavailable (or different) at prediction time, or that leaks the label. Classic examples: using a value computed after the event ("total lifetime spend" when predicting a mid-lifetime event), or a feature derived from the target. Leakage makes offline metrics look amazing and the model fail in production. Guard against it with point-in-time-correct feature generation (build features "as of" the event timestamp) and time-based validation splits.

Training-serving skew — the same feature computed differently offline vs online (a different window, default, or code path). It's silent and it's the #1 production ML failure — it has its own concept later in the track. The structural fix is to compute each feature once and serve it consistently.

Feature stores (why they exist)

A feature store is the system that makes features consistent and reusable. It computes a feature from a single definition and serves it to both training and serving, eliminating skew by construction, and it lets teams share features instead of re-implementing them. Two halves:

Offline store — historical feature values (in a warehouse) for building training sets, with point-in-time joins.
Online store — the latest feature values in a low-latency store (e.g. Redis) for sub-10ms retrieval at serving.

You'll go deeper on feature stores (and their at-scale challenges) later in the track; here, just anchor why they exist: consistency, reuse, and freshness.

A quick worked example

Predicting churn: raw data is a signup date and an event log. Good engineered features: account_age_days, sessions_last_7d, sessions_last_30d, days_since_last_session, avg_session_minutes_30d, sessions_7d ÷ sessions_30d (recency-weighted engagement trend). Note every aggregation names a window — and every window must be computed identically in training and serving, "as of" the prediction time, or you've introduced skew or leakage.

Common interview mistakes

Mistake 1: Ignoring leakage. A feature that peeks at the future or the label produces beautiful offline metrics and a broken production model.

Mistake 2: One-hot encoding high-cardinality categoricals. Millions of columns; use target encoding or embeddings instead.

Mistake 3: Fitting scalers/encoders on all data. Fit on training only, and ship the transform with the model — otherwise you leak and skew.

Mistake 4: Skipping feature consistency in system design. Any train-offline / serve-online system must address training-serving skew (feature store).

Mistake 5: Underestimating the effort. Data + features are 60–80% of real ML work, not a preprocessing footnote.

Key vocabulary

Feature — A transformed model input derived from raw data; feature quality caps model performance.
Aggregation feature — A windowed statistic (count/avg/sum over N days); the main source of skew/leakage bugs.
Encoding — Turning categoricals into numbers (one-hot, target/frequency encoding, embeddings) based on cardinality.
Data leakage — Using information unavailable at prediction time (or the label) in a feature; inflates offline metrics.
Point-in-time correctness — Building features "as of" the event time so training matches serving.
Feature store — A compute-once/serve-consistently system (offline + online) that eliminates training-serving skew.