Computer Vision with Convolutional Neural Networks
Understand convolutions from the arithmetic up, build CNNs from LeNet to ResNet in PyTorch, and train, transfer and evaluate them for classification, detection and segmentation.
Intermediate · Machine Learning · about 17 hours · 5 modules · certificate on completion
Taught by Aïssatà Coulibàly, Priyà Nairé, AI Learning Companions.
About this course
Convolutional neural networks are how machines read images, from phone apps that spot crop disease to systems that help doctors screen scans. This course takes them apart and rebuilds them in PyTorch and torchvision: what a convolution computes and how big its output is, why architectures moved from LeNet to VGG and ResNet, and how to make a network that does well on its test set also work on the images it will meet in the field.
You'll implement convolution as a matrix multiplication, measure what residual connections do to gradients, train on studio images and test on field images to see which augmentations close the gap, and transfer a pretrained network to new classes from a handful of labels. Then you'll score detections with average precision and train a small U-Net. Every image is generated on your own machine, so the labs run on a laptop CPU. The capstone asks a leaf disease classifier that has only seen studio photos to work in the field. The course builds on Deep Learning with PyTorch and goes further than the computer vision modules of Applied Machine Learning with TensorFlow.
What you'll learn
- Calculate convolution output sizes, parameter counts and receptive fields, and implement a convolution as a matrix multiplication
- Build and compare CNNs from LeNet and VGG to ResNet and depthwise separable designs in PyTorch
- Load image data with torchvision, and design augmentation that matches the shift between training and deployment images
- Transfer pretrained networks to new classes by feature extraction and fine-tuning
- Evaluate classifiers, detectors and segmenters with per-class metrics, calibration, mAP, IoU and Dice
Skills you'll gain
Convolutional neural networks, CNN architectures, Residual networks, Data augmentation, Domain shift, Transfer learning, Object detection, Image segmentation, Model evaluation, PyTorch and torchvision
Tools
Python 3, PyTorch, torchvision, NumPy, Pillow, scikit-learn
Before you start
Deep Learning with PyTorch, or equivalent experience writing PyTorch training loops with Dataset, DataLoader and nn.Module. Install Python 3.10 or later, PyTorch 2.6 or later and the matching torchvision (pip install torch torchvision, or the selector at pytorch.org; the CPU build is enough). All images are generated by the course's scripts, so nothing is downloaded except, optionally, ImageNet weights for ResNet-18 (about 45 MB) in the capstone.
Modules
Convolutions from first principles
Turn images into tensors, then work out exactly what a convolution computes, how big its output is, how far each unit sees and what pooling adds.
- Images as tensors (10 min)
- The convolution operation (12 min)
- Receptive fields, pooling and invariance (12 min)
- Hands-on lab: Convolution by hand, checked against PyTorch
CNN architectures from LeNet to ResNet
Follow the ideas that made CNNs deeper and better, from LeNet's conv-pool pattern through VGG and Inception to residual networks and depthwise separable convolutions, and build each in PyTorch.
- LeNet and the classic CNN pattern (10 min)
- AlexNet, VGG and Inception: depth, small kernels and 1×1 convolutions (12 min)
- Residual networks (12 min)
- Efficient convolutions (10 min)
- Hands-on lab: Build LeNet, a VGG-style network and a ResNet
Training CNNs: data, augmentation and shift
Load image datasets without leaking between splits, augment them with torchvision's v2 transforms, and find out how a model trained on tidy images copes with the ones it will really meet.
- Image datasets and loaders (12 min)
- Data augmentation with torchvision transforms (12 min)
- When test images don't look like training images (12 min)
- Hands-on lab: Train in the studio, test in the field
Transfer learning
Reuse networks pretrained on large datasets: load them and their preprocessing from torchvision, then adapt them by feature extraction or fine-tuning, and avoid the ways transfer goes wrong.
- Pretrained models in torchvision (12 min)
- Feature extraction and fine-tuning (12 min)
- When transfer goes wrong (10 min)
- Hands-on lab: Transfer a CNN to new classes with few labels
Detection, segmentation and honest evaluation
Evaluate classifiers beyond accuracy, then move from labelling images to locating and outlining what's in them: detectors and their mAP, and segmenters with IoU and Dice.
- Evaluating image classifiers (12 min)
- Object detection (12 min)
- Measuring detectors: precision, recall and mAP (10 min)
- Semantic and instance segmentation (12 min)
- Hands-on lab: Score detections with AP, then measure a segmenter
Project: Diagnose leaf disease from field photos
make_leaves.py generates synthetic 64 x 64 leaf photos in ImageFolder layout, in four classes: healthy, leaf_spot, blight and mosaic. train, val and test hold studio photos (one centred leaf on a plain light background in even light: 1,600, 400 and 400 images); field holds 400 photos like the ones farmers take (soil and grass behind the leaf, any angle and size, uneven light, blur, noise) for the final evaluation only, and field_val 100 more for choosing between models. Train only on studio photos and make the classifier work in the field. On the course's machine a small ResNet-style network trained without augmentation scored 0.998 on the studio test set and about 0.27 in the field, calling almost every leaf blight; geometric and photometric augmentation raised field accuracy to about 0.54, with almost no healthy field leaves recognised. Close the gap with what the course teaches: augmentation that simulates the field's backgrounds (for example, cut each studio leaf out of its plain background by colour and paste it onto random textures), stronger photometric changes, a better architecture, and optionally an ImageNet-pretrained ResNet-18 (about 45 MB to download) with its own preprocessing. Evaluate honestly, and report what still fails.
Part of
The products this course names are trademarks of their owners, who haven't made or endorsed it.