Classical Classification and Clustering
Apply and tune logistic regression, k-NN, SVMs, decision trees and ensembles, then find structure in unlabelled data with k-means and PCA, all in scikit-learn.
Intermediate · Machine Learning · about 16 hours · 5 modules · certificate on completion
Taught by Marisolò Garcíà, Zawadì Mutembò, AI Learning Companions.
About this course
This course is a practical tour of the classical algorithms that still solve most tabular problems, each applied, tuned and understood in scikit-learn. You start with logistic regression, reading its coefficients as odds ratios and controlling it with C and L1 or L2 penalties, and k-nearest neighbours, with its distance metrics, k and weights, and compare classifiers properly with ROC curves, AUC and shared cross-validation folds. Then come support vector machines: maximum margins, support vectors, the kernel trick, the tuning of C and gamma, and calibrated probabilities. Decision trees follow, with information gain and pruning, and then the ensembles that fix their weaknesses: bagging, random forests with out-of-bag scores and permutation importance, gradient boosting, voting and stacking.
The last two modules turn to unlabelled data. You cluster with k-means, choose K with the elbow method and silhouette scores, and learn where DBSCAN or hierarchical clustering fit better; then you reduce dimensions with PCA, choosing components by explained variance, compressing data and mapping it in two dimensions beside t-SNE. Every lesson has a JupyterLab walkthrough with real outputs, and every module a lab on data such as scikit-learn's wine and digits sets and a made-up football dataset, where clustering rediscovers playing styles. The capstone is an end-to-end pipeline that compares classifier families by cross-validation and selects the best one.
What you'll learn
- Train and tune logistic regression and k-NN classifiers, and compare classifiers with ROC AUC and cross-validation
- Apply support vector machines with linear, polynomial and RBF kernels, tuning C and gamma
- Build, prune and read decision trees, and beat them with random forests and gradient boosting
- Cluster unlabelled data with k-means, choosing K with the elbow method and silhouette scores
- Reduce dimensions with PCA to compress data, visualise it and speed up models
- Build an end-to-end pipeline that selects the best classifier by cross-validation
Skills you'll gain
Logistic regression, k-nearest neighbours, Support vector machines, Decision trees, Random forests, Gradient boosting, k-means clustering, Principal component analysis, Model selection, scikit-learn
Tools
JupyterLab, Python 3, scikit-learn, NumPy, pandas, matplotlib
Before you start
Machine Learning Foundations with Python, or Foundations of Machine Learning (Vietnamese and English): train/test splits, fitting scikit-learn models and basic metrics. Comfortable Python with pandas and NumPy. Machine Learning from Scratch is a good companion: it builds several of these algorithms by hand. Install Python 3 with NumPy, pandas, scikit-learn, matplotlib and JupyterLab; every dataset ships with scikit-learn or the labs.
Modules
Logistic Regression & KNN
Two classic classifiers from opposite traditions: logistic regression, which turns a weighted sum into a probability, and k-nearest neighbours, which votes among similar examples. Tune both, read what they learned, and compare them fairly with ROC AUC.
- Logistic regression: probabilities, log-odds and regularisation (12 min)
- k-nearest neighbours: distance metrics, k and weights (12 min)
- Comparing classifiers: ROC curves and AUC (12 min)
- Hands-on lab: Classify wines with logistic regression and k-NN
Support Vector Machines
Separate classes with the widest possible margin, let a few support vectors define the boundary, and bend it with kernels: tune C and gamma, and turn SVM scores into calibrated probabilities.
- Maximum margins and support vectors (12 min)
- The kernel trick (12 min)
- Tuning C and gamma, and getting probabilities (12 min)
- Hands-on lab: Bend the boundary with kernels
Decision Trees & Ensembles
Grow interpretable trees and prune them, then combine many trees: bagging and random forests to reduce variance, gradient boosting to reduce bias, and voting and stacking to mix different models.
- Decision trees: information gain and pruning (12 min)
- Random forests: bagging and random features (12 min)
- Boosting, voting and stacking (12 min)
- Hands-on lab: From one tree to a forest
K-Means Clustering
Discover groups in unlabelled data with centroid-based clustering: run k-means well, choose K with the elbow method and silhouette scores, profile the clusters, and know when another method fits better.
- How k-means works, and how to use it well (12 min)
- Choosing K: the elbow and the silhouette (12 min)
- When k-means fails, and alternatives (10 min)
- Hands-on lab: Find playing styles with k-means
PCA & Dimensionality Reduction
Reduce many features to a few with principal component analysis: see what the components are, choose how many to keep, compress and reconstruct data, speed up models, and map high-dimensional data in two dimensions.
- What PCA finds (12 min)
- Choosing components, compressing data and pipelines (12 min)
- Seeing high-dimensional data (10 min)
- Hands-on lab: Compress the digits with PCA
Project: End-to-end classification pipeline
Build a complete, reproducible pipeline that chooses the best classifier for scikit-learn's breast cancer data (569 tumours, 30 measurements each, with 1 meaning malignant in the starter file). Split off a stratified test set first and don't touch it again until the end. Put the preprocessing (scaling, and PCA if you try it) inside each model's pipeline. Compare at least four classifier families from this course, including logistic regression, an SVM and a random forest, with cross_validate on the same StratifiedKFold folds, reporting the mean and spread of ROC AUC, F1 and recall for the malignant class. Choose the most promising model, tune its hyperparameters with GridSearchCV (or RandomizedSearchCV) on the training set, then score the tuned pipeline once on the test set: a confusion matrix, a classification report and ROC AUC. Decide, and justify, a decision threshold for a screening setting, where a missed malignant tumour costs far more than a false alarm. Report what you found, including what didn't work.
Part of
The products this course names are trademarks of their owners, who haven't made or endorsed it.