Introduction to Neural Networks and TensorFlow
Build neural networks from the single neuron up: code a perceptron and backpropagation in NumPy, then train and evaluate feed-forward networks with TensorFlow and Keras.
Intermediate · Machine Learning · about 13 hours · 5 modules · certificate on completion
Taught by Jae-Wòn Parkk, Mei-Líng Zhāng, AI Learning Companions.
About this course
Neural networks power modern image recognition, speech and language models, and they're built from a simple part: a neuron that weighs its inputs, adds them up and passes the result through an activation function. This course takes you from that single neuron to working deep learning code. You'll write a perceptron in NumPy and see exactly why one layer can't learn XOR, add hidden layers and non-linear activations to build multi-layer perceptrons, and derive backpropagation yourself, checking your gradients against numerical ones before you train a network from scratch.
Then you'll meet TensorFlow and its high-level Keras API: tensors, automatic differentiation with GradientTape, graphs with tf.function, and models you build, compile, fit and evaluate in a few lines. You'll finish by training a handwritten-digit classifier, with a softmax output, cross-entropy loss, validation curves and a confusion matrix, and the capstone asks you to build, improve and evaluate an MLP on the full MNIST dataset. Every lab runs on an ordinary laptop with fixed random seeds, and comes with starter code and the output to expect. It leads on to Training Deep Neural Networks.
What you'll learn
- Explain how an artificial neuron and a perceptron work, and why a single layer can't learn XOR
- Describe the architecture of a multi-layer perceptron, count its parameters and choose its activation functions
- Derive and code backpropagation for a small network, and prove it right with a gradient check
- Build, train and evaluate feed-forward networks with TensorFlow tensors, GradientTape and the Keras Sequential API
- Choose output activations and loss functions for regression, binary and multi-class tasks
- Train a handwritten-digit classifier and judge it with validation curves and a confusion matrix
Skills you'll gain
Neural networks, Perceptrons, Multi-layer perceptrons, Activation functions, Backpropagation, Gradient descent, TensorFlow, Keras, Image classification
Tools
Python 3, NumPy, TensorFlow, Keras, scikit-learn
Before you start
Classical Classification and Clustering, or Machine Learning Foundations with Python: train/test splits, accuracy and scikit-learn classifiers. Comfortable Python and NumPy, and what a derivative is. Install Python 3.10 or later, then pip install tensorflow scikit-learn (TensorFlow 2.16 or later includes Keras 3). The capstone downloads MNIST, about 11 MB, once.
Modules
The Perceptron and Biological Inspiration
Meet the artificial neuron and the perceptron: how a weighted sum and a step function make a linear classifier, how the perceptron learns from its mistakes, and why a single layer can never learn XOR.
- From biological neurons to artificial ones (10 min)
- The perceptron: a linear classifier that learns from mistakes (12 min)
- The XOR problem and the limits of a single layer (12 min)
- Hands-on lab: Build a perceptron and watch it fail on XOR
Multi-Layer Perceptrons (MLPs)
Add hidden layers of fully connected neurons, see why they need non-linear activation functions, compute forward passes with matrices, and learn what the universal approximation theorem does and doesn't promise.
- Hidden layers and fully connected networks (12 min)
- Why activation functions must be non-linear (12 min)
- The universal approximation theorem (10 min)
- Hands-on lab: Solve XOR and the two moons with hidden layers
Training with Backpropagation
Follow one training step from start to finish: a forward pass that makes predictions, a loss that measures the error, a backward pass that applies the chain rule to get every gradient, and an update that moves the weights.
- The forward pass and loss functions (12 min)
- The chain rule and the backward pass (14 min)
- Gradient descent: updating the weights (12 min)
- Hands-on lab: Backpropagation from scratch
TensorFlow and Keras Basics
Meet TensorFlow and its high-level Keras API: tensors and their shapes and types, automatic gradients with GradientTape, computation graphs with tf.function, and models you build, compile, fit and evaluate in a few lines.
- Tensors: shapes, types and operations (12 min)
- Automatic differentiation and computation graphs (12 min)
- Building models with Keras (14 min)
- Hands-on lab: Tensors, gradients and a first Keras model
Building a Digit Classifier
Put it all together on handwritten digits: scale and flatten the images, give the network a softmax output and a categorical cross-entropy loss, watch validation accuracy as it trains, and evaluate and save the finished classifier.
- Images as input: scaling and flattening (12 min)
- Softmax outputs and categorical cross-entropy (12 min)
- Monitoring validation accuracy and evaluating the classifier (12 min)
- Hands-on lab: Classify handwritten digits with Keras
Project: Digit classifier with TensorFlow
Build, train and evaluate a multi-layer perceptron that classifies the handwritten digits of MNIST with TensorFlow and Keras. Start from mnist_starter.py: it downloads MNIST (about 11 MB, once, into ~/.keras/datasets), scales the pixels and trains a baseline with one hidden layer of 128 units, which reaches a validation accuracy of about 0.976 in 10 epochs. Improve on it systematically. Try at least three variations, such as wider or deeper hidden layers, ReLU against tanh, a different batch size or learning rate, dropout or early stopping; change one thing at a time, keep the random seed fixed, and compare them on validation accuracy. Choose one, and only then evaluate it on the 10,000 test images, once: a well-built MLP reaches about 0.98. Study its mistakes with a confusion matrix and some misclassified images, save it in the .keras format, and write a predict.py that loads it and classifies test images.
Part of
The products this course names are trademarks of their owners, who haven't made or endorsed it.