Machine Learning Design Patterns
Solve the recurring problems of ML systems with proven patterns for representing data, framing problems, training, serving and reproducibility, and know what each one costs.
Intermediate · AI Engineering · about 25 hours · 5 modules · certificate on completion
Taught by Hiroshì Yamadà, Wei-Chéng Liú, AI Learning Companions.
About this course
Machine learning systems keep running into the same problems: a category with too many values to one-hot encode, a rare group of cases drowned out by the common ones, a week of training lost to a crashed machine, features computed one way in training and another in production, results nobody can reproduce. Design patterns are the field's accumulated answers, each a named problem with a proven solution and known trade-offs. This course teaches the core patterns from the catalogue made popular by Machine Learning Design Patterns (O'Reilly, 2020), in five groups: repeatable splitting and how to choose patterns; hashed features, embeddings, feature crosses and multimodal input; reframing, multilabel, ensembles and cascades; useful overfitting, checkpoints, transfer learning and distribution strategies; and stateless serving, batch serving, workflow pipelines and feature stores.
Every pattern is built and measured, not just described. You'll compare hashing with one-hot encoding as collisions rise, learn product embeddings from shopping baskets, build a cascade for catering orders and test it against a single flexible model, make an interrupted training run resume to the same weights, measure what a naive feature join leaks compared with a point-in-time ASOF join in DuckDB, and serve the model through a stateless, validated FastAPI function with keyed predictions, a batch job and a cached workflow pipeline. The capstone hands you a working but badly designed churn pipeline to refactor with the patterns it needs.
What you'll learn
- Apply established design patterns to recurring ML problems, choosing each from a symptom and stating its trade-offs
- Represent high-cardinality, interacting and multimodal data with hashed features, embeddings, feature crosses and multimodal inputs
- Frame problems with reframing, multilabel outputs, ensembles and cascades
- Make training robust and efficient with checkpoints, transfer learning and distribution strategies
- Design resilient serving with stateless functions, keyed predictions and batch serving
- Make pipelines reproducible with repeatable splits, workflow pipelines and point-in-time feature joins
Skills you'll gain
ML design patterns, Feature hashing, Embeddings, Feature crosses, Cascades and ensembles, Checkpointing, Transfer learning, Model serving, Point-in-time joins, Workflow pipelines
Tools
Python 3, scikit-learn, pandas, DuckDB, FastAPI, pytest
Before you start
Foundations of AI System Design, or equivalent experience framing, building and evaluating models with pandas and scikit-learn. Install Python 3.10 or newer, then: pip install pandas numpy scipy scikit-learn duckdb pyarrow joblib fastapi uvicorn httpx pytest. Every dataset is generated by the labs or ships with scikit-learn.
Modules
Introduction to ML Patterns
What a design pattern is, why machine learning systems need their own, how the catalogue is organised, and how to choose a pattern from a symptom, practised on your first pattern: repeatable splitting.
- What a design pattern is (10 min)
- Why ML systems need patterns (12 min)
- Choosing and applying patterns (12 min)
- Hands-on lab: Your first pattern: repeatable splitting
Data Representation Patterns
Represent inputs so models can use them: hash high-cardinality categories, learn embeddings, cross features to capture interactions, and combine text, categories and numbers in one model.
- Hashed feature (12 min)
- Embeddings (12 min)
- Feature cross (12 min)
- Multimodal input (12 min)
- Hands-on lab: Represent data four ways
Problem Representation Patterns
Frame the learning problem to fit the decision: reframe outputs, predict several labels at once, combine models in ensembles, and split a problem into a cascade of models.
- Reframing (12 min)
- Multilabel (10 min)
- Ensembles (12 min)
- Cascade (12 min)
- Hands-on lab: Reframe, multilabel, ensemble and cascade
Training Optimisation Patterns
Patterns for the training loop: overfit on purpose when the input space is closed, checkpoint so long runs survive failures, transfer what a model has learnt to a new task, and distribute training across workers.
- Useful overfitting (11 min)
- Checkpoints (13 min)
- Transfer learning (12 min)
- Distribution strategy (13 min)
- Hands-on lab: Overfit, checkpoint, transfer and parallelise
Resilient Serving and Reproducibility
Serve models with stateless functions and batch jobs, make the process repeatable with workflow pipelines, and build training data with point-in-time joins, the core of the feature store pattern.
- Stateless serving function (13 min)
- Batch serving (11 min)
- Workflow pipeline (12 min)
- Feature store (14 min)
- Hands-on lab: Serve, batch-score, pipeline and join without leaks
Project: Refactor a churn pipeline with design patterns
A subscription company's churn model was built in a hurry. legacy_train.py joins each label to the customer's latest feature snapshot, one-hot encodes a high-cardinality referrer site, fits a scaler on all rows before a random, unseeded split, trains for 30 long epochs without checkpoints, and pickles the classifier without its preprocessing. legacy_serve.py rebuilds the preprocessing by hand with means copied from a notebook, counts each customer's calls in memory, and fails on referrers it hasn't seen; legacy_batch.py calls it one customer at a time and swallows the errors. Run them first: the reported test AUC changes from run to run and the nightly job fails for about half of the new customers. Refactor the system: keep the legacy files as the "before", and build a workflow pipeline whose steps create a point-in-time training set, split it repeatably, train a single exported pipeline (with a hashed referrer feature) with checkpoints, test and serve it through a stateless, validated FastAPI function with keyed predictions and a model version, and batch-score new customers with the shared prediction code.
Part of
The products this course names are trademarks of their owners, who haven't made or endorsed it.