Transformers and Large Language Models

Build attention and transformer blocks from scratch, understand how large language models are pretrained, aligned and scaled, and fine-tune, decode and evaluate them with Hugging Face.

Advanced · Generative AI · about 23 hours · 5 modules · certificate on completion

Taught by Alekseï Volkovì, Björn Sigurdssòn, AI Learning Companions.

About this course

Large language models are transformers trained on enormous amounts of text, and this course opens them up. You'll implement scaled dot-product attention, multi-head attention, layer normalisation, a complete transformer block and positional encodings in NumPy, and check every piece against PyTorch. Then you'll compare how models are pretrained, by causal language modelling like GPT or by masking and denoising like BERT, T5 and BART, train a tiny GPT yourself, and follow a model through instruction tuning and preference alignment with RLHF and DPO, with the scaling laws that guide how big to build and how long to train.

The second half is hands-on with Hugging Face: fine-tune DistilBERT for sentiment with Datasets and the Trainer API and prepare labels for named entity recognition; write greedy, beam search and sampling decoders from scratch before using generate(); and apply sequence-to-sequence models to summarisation and extractive question answering, scored with ROUGE, exact match and F1 that you implement yourself. Every module has a lab with exact expected output wherever the maths allows it, and the capstone fine-tunes a summariser for a domain of your choice and finds the decoding strategy you'd ship. The code works with transformers 4.46 and later, including version 5. It builds on Text Representation: From Vectors to Context and Statistical NLP: Models, Parsing and Information Retrieval.

What you'll learn

  • Explain and implement self-attention, multi-head attention, positional encoding and the transformer block from scratch
  • Describe causal and masked pretraining, instruction tuning, preference alignment with RLHF and DPO, and scaling laws
  • Fine-tune pre-trained transformers for text classification, and prepare data for named entity recognition
  • Implement and compare greedy, beam search, temperature, top-k and nucleus sampling decoding
  • Adapt sequence-to-sequence models for summarisation and extractive QA, and evaluate them with ROUGE, exact match and F1

Skills you'll gain

Transformer architecture, Self-attention, Language model pretraining, Fine-tuning, Hugging Face Transformers, Text generation and decoding, Summarisation, Question answering, Model evaluation, PyTorch

Tools

Python 3, PyTorch, NumPy, Hugging Face Transformers, Hugging Face Datasets, rouge-score

Before you start

Text Representation: From Vectors to Context and Statistical NLP: Models, Parsing and Information Retrieval, or the same skills; comfort with PyTorch (Deep Learning with PyTorch covers it), matrix multiplication, softmax and cross-entropy. You need Python 3.10 or later with PyTorch, transformers 4.46 or later, datasets and rouge-score; the labs download small models, up to about 350 MB each. A GPU helps for the capstone but isn't required.

Modules

  1. Transformer architecture deep dive

    Take the transformer apart: scaled dot-product attention and its masks, multi-head attention, positional encodings, the feed-forward, residual and normalisation layers of a block, and how blocks stack into encoders, decoders and encoder-decoders.

    • Scaled dot-product attention (14 min)
    • Multi-head attention and positional encoding (14 min)
    • The transformer block: feed-forward, residuals and layer normalisation (12 min)
    • Encoder, decoder and encoder-decoder stacks (12 min)
    • Hands-on lab: Build a transformer block in NumPy

  2. LLMs and masked LMs

    Compare the objectives that pretrain language models, causal language modelling for GPT-style decoders and masked or denoising objectives for BERT, T5 and BART, then follow a model from pretraining through fine-tuning and preference alignment, and see what scaling laws say.

    • Causal language modelling (12 min)
    • Masked and denoising objectives (12 min)
    • From pretraining to fine-tuning and alignment (14 min)
    • Scaling laws, in outline (12 min)
    • Hands-on lab: Pretraining objectives in code

  3. Text classification pipeline

    Fine-tune a pre-trained transformer the Hugging Face way: load and prepare data with Datasets, tokenise for the model, train and evaluate with the Trainer API, and extend the same pipeline to named entity recognition.

    • Hugging Face Datasets (12 min)
    • Tokenisation for transformers (12 min)
    • Fine-tuning with the Trainer API (14 min)
    • Evaluating classifiers and NER (12 min)
    • Hands-on lab: Fine-tune DistilBERT and prepare NER labels

  4. Text generation and decoding

    Generate text one token at a time and choose how: greedy and beam search, sampling with temperature, top-k and top-p, written from scratch on a toy model and then used through Hugging Face's generate().

    • Autoregressive generation (12 min)
    • Greedy and beam search (12 min)
    • Sampling: temperature, top-k and top-p (12 min)
    • Choosing and evaluating a decoding strategy (12 min)
    • Hands-on lab: Decoding strategies from scratch and with generate()

  5. Summarisation and question answering

    Apply transformers to two classic tasks: abstractive summarisation with sequence-to-sequence models such as BART and T5, scored with ROUGE, and extractive question answering, scored with exact match and F1.

    • Sequence-to-sequence models: BART and T5 (12 min)
    • Measuring summaries with ROUGE (12 min)
    • Extractive question answering (12 min)
    • Scoring QA: exact match and F1 (12 min)
    • Hands-on lab: Summarise, answer questions and score them

Project: Custom fine-tuned summariser

Fine-tune a sequence-to-sequence model to summarise texts from one domain, evaluate it rigorously, and choose how to decode it. Start from the starter script train_summariser.py, which fine-tunes t5-small on DialogSum (knkarthick/dialogsum on the Hugging Face Hub: dialogues with human-written summaries, licensed CC BY-NC-SA 4.0), or swap in another domain's document and summary pairs, such as BillSum (US legislation) or your own. Measure the pretrained model first, then fine-tune it: a few thousand examples and two or three epochs are feasible on a laptop CPU in a few hours and far faster on a GPU. Build an evaluation pipeline that reports ROUGE-1, ROUGE-2 and ROUGE-L on a held-out test set, then compare decoding strategies on the same test examples: greedy decoding, beam search with two widths and two length penalties, and nucleus sampling at two temperatures with fixed seeds. Read the summaries as well as the scores: look for invented facts, omissions and repetition, and say which strategy you'd ship and why. In a quick check of the starter script (64 training dialogues, one epoch, 16 validation dialogues), ROUGE-1 rose from 0.23 to 0.34, so a full run should do clearly better.

Part of

The products this course names are trademarks of their owners, who haven't made or endorsed it.

Start the course free