MLOps Foundations: Experiments, Models and Reproducibility with MLflow
Track every experiment, package models with signatures, promote them with registry aliases and rebuild any training run, using MLflow on your own laptop.
Intermediate · AI Engineering · about 16 hours · 5 modules · certificate on completion
Taught by Chukwuemekà Obiechè, Ismaìlè Rahimovì, AI Learning Companions.
About this course
Most models that fail in production fail for operational reasons: nobody knows which data or code produced them, the package that ran in a notebook won't load in a service, or a change can't be rolled back. This course teaches the habits and tooling that prevent those failures, using MLflow, the open-source platform for the ML lifecycle, entirely on your own machine with a local tracking server and a SQLite backend.
You'll log experiments that a reviewer can trust, organise sweeps and choose winners with queries rather than by eye, package models with signatures and custom pyfunc logic, and serve them over REST. You'll then run a registry the way production teams do, promoting through a gate in code and releasing or rolling back by moving aliases, and finish by making training reproducible with seeds, dataset digests, pinned environments and MLflow Projects. Every module has a lab whose code was run with MLflow 3, and the capstone takes a churn model through the whole lifecycle. It prepares you for Deploying and Serving Machine Learning Models and Monitoring Models in Production: Drift, Data Quality and Alerts.
What you'll learn
- Run an MLflow tracking server with a SQLite backend and log parameters, metrics, tags and artifacts to it
- Organise sweeps as nested runs and choose models with search_runs on validation metrics
- Package models with signatures, input examples and custom pyfunc logic, and serve them over REST
- Release and roll back models by moving registry aliases through a promotion gate written in code
- Make training reproducible with seeds, dataset digests, pinned environments and MLflow Projects
Skills you'll gain
MLOps, Experiment tracking, MLflow, Model packaging, Model registry, Reproducible training, Model versioning, Python
Tools
MLflow, scikit-learn, pandas, SQLite, Git, curl
Before you start
Machine Learning Foundations with Python, or equivalent experience training and evaluating scikit-learn models, and comfort in a terminal (Linux Command Line Essentials covers it). You need Python 3.10 or later, Git and a browser; the course installs MLflow 3, scikit-learn and pandas with pip. No cloud account is needed.
Modules
The ML lifecycle and a tracking server
What MLOps adds to model building, how MLflow's parts fit together, and a local tracking server with a SQLite backend that every later lab logs to.
- What MLOps adds to machine learning (10 min)
- MLflow's building blocks (12 min)
- Running MLflow locally: tracking URIs and the server (12 min)
- Hands-on lab: Start a tracking server and log your first runs
Experiment tracking: parameters, metrics and artifacts
Log what a reviewer needs to trust a result, organise a sweep as nested runs, see what autologging does and doesn't capture, and choose a winner with search_runs instead of by eye.
- Logging parameters, metrics and tags (12 min)
- Artifacts: evidence a reviewer can check (10 min)
- Nested runs, autologging and search (13 min)
- Hands-on lab: Track a sweep with nested runs and choose a winner
Packaging models: flavours, signatures and pyfunc
What a logged model contains, how a signature guards its inputs, how to wrap preprocessing and business rules in a custom pyfunc model, and how to score a packaged model locally.
- Inside an MLflow model (11 min)
- Signatures and input examples (12 min)
- Custom models with pyfunc (12 min)
- Scoring a packaged model locally (10 min)
- Hands-on lab: Package a model with a signature and a threshold, then serve it
The Model Registry: versions, aliases and promotion
Register models as numbered versions, release and roll back by moving aliases, and promote only through a gate in code that records why each version was accepted or refused.
- Registered models and versions (10 min)
- Aliases: champion, challenger and rollback (12 min)
- A promotion gate in code (12 min)
- Hands-on lab: Register, gate and promote with aliases
Reproducibility and MLflow Projects
Why the same code can give a different model, how to record data and environments with each run, how MLflow Projects give training a reproducible entry point, and how to prove a run can be rebuilt.
- Why the same code gives a different model (10 min)
- Versioning data and environments (12 min)
- MLflow Projects (12 min)
- Reproduce a run from its record (10 min)
- Hands-on lab: Make training reproducible, and prove it
Project: Take a churn model through the MLflow lifecycle
A subscription business wants to predict which customers will cancel next month, so its retention team can call the riskiest ones first. A previous model was trained in a notebook nobody can rerun. Build the replacement the MLOps way, on the synthetic data from make_churn_data.py (2,000 customers with tenure, charges, support calls, late payments, contract and payment method). Make it an MLflow Project in a Git repository with entry points to prepare data, train one model and run a sweep; track every run with parameters, validation metrics, the dataset input and its hash, and evidence artifacts; package the chosen model as a custom pyfunc model that one-hot encodes the categories, applies a threshold the team can afford to call (about the riskiest 20% of customers), and has a signature and an input example; register candidates, promote one through a gate script that records its decisions, and show a rollback; and finally prove the champion can be rebuilt from its record.
Part of
The products this course names are trademarks of their owners, who haven't made or endorsed it.