What training actually does
Training adjusts a model's weights to minimize a loss function (how wrong its predictions are) via gradient descent: forward pass (predict) → compute loss → backpropagation (gradients) → update weights, repeated over the data in epochs. You don't need to implement it, but you must reason about it.
The core dials
- Learning rate — step size. Too high → diverges/unstable; too low → slow, stuck. The most important hyperparameter.
- Batch size — examples per update; affects stability and memory.
- Epochs — passes over the data; too few underfits, too many overfits.
- Regularization (dropout, weight decay) — fights overfitting.
Overfitting vs underfitting (the central tension)
- Underfitting — model too simple / trained too little → bad on both train and validation. Fix: train more, bigger model, better features.
- Overfitting — model memorizes the training data → great on train, bad on validation/new data. Fix: more data, regularization, early stopping, simpler model.
- You diagnose this with a train/validation split: watch both losses. Train loss ↓ while validation loss ↑ = overfitting. Early stopping halts at the best validation point.
Data is the lever
For fine-tuning especially, data quality dominates: a few hundred clean, correct, diverse, representative examples beat thousands of noisy ones. Curate carefully, dedupe, cover edge cases, and hold out a clean test set the model never sees. Class imbalance, label errors, and leakage (test info sneaking into training) are the usual killers.
Evaluation is not optional
Always evaluate on a held-out set the model didn't train on — training metrics lie (a memorizing model looks perfect on train). This is where the Evaluation course connects: define the metric that matches the task, and judge the model on data it's never seen.