Machine Learning
Where Foundations holds the model-agnostic concepts, this branch is the practical classical toolkit: the algorithms, the supervised workflow, and the habits that decide whether a model is trustworthy. Most of it predates deep learning — and for tabular data it still wins, ships faster, and is far easier to debug than a neural net.
Rule of thumb: reach for a gradient-boosted tree before a neural network on tabular data. Start simple; earn complexity.
Mental model
Statistical machine learning selects a hypothesis from data under a loss, regularization, and validation protocol. The algorithm matters, but representation, leakage control, calibration, and error analysis usually determine whether measured performance survives deployment.
Roadmap: workflow and algorithms
- The supervised learning workflow, end to end
- Error analysis: reading your model's mistakes
- Pipelines & preventing preprocessing leakage
Core algorithms
- Linear & logistic regression
- Decision trees & ensembles (RF, gradient boosting)
- kNN & SVM: distance and margins
- Clustering & PCA: learning without labels
Making models work
- Feature engineering
- Regularization: L1, L2 & how they differ
- Cross-validation done right
- Handling class imbalance
- Hyperparameter tuning
Connects to: Learning Foundations · Data for AI · Evaluation and Measurement
Core sources
- An Introduction to Statistical Learning — theory, algorithms, and reproducible labs.
- The Elements of Statistical Learning — deeper treatment of supervised and unsupervised methods.
- scikit-learn User Guide — canonical implementation semantics and model-selection guidance.
- Pattern Recognition and Machine Learning — probabilistic framing of classical ML methods.
- StatQuest — trees, boosting, and ROC/PR intuition.