Building RAG and Agentic Systems
Build retrieval-augmented and agentic AI systems that hold up: measured retrieval, hybrid search and re-ranking, tool-using agents with memory and plans, and multi-agent teams that check each other's work.
Advanced · Generative AI · about 22 hours · 5 modules · certificate on completion
Taught by Luís Ferreiràs, Layla Al-Mansourì, AI Learning Companions.
About this course
Retrieval-augmented generation and agents are how language models get to work on real knowledge and real tasks, and how they most often fail in production: the right passage isn't retrieved, a tool is called with invented arguments, an agent loops until the budget runs out, or one agent's mistake is repeated confidently by the next. This course builds both kinds of system from the parts up, measuring each part as you go. You'll index a handbook and measure retrieval with recall@k and MRR, compare exact and approximate vector search, then improve retrieval with chunking, BM25, hybrid fusion, re-ranking and metadata filters. You'll build an agent loop that validates inputs, retries what's safe to retry, asks before acting and stops itself; give agents memory and plans they check and repair; and finish with a researcher, writer and reviewer team that won't sign off an unsupported sentence.
The labs run on a laptop without an API key, on recorded model turns, so every check is exact, and each has live steps for the Claude API and optional steps with open embedding and re-ranking models. The capstone is an autonomous research assistant: a Researcher agent that calls retrieval and web search tools, and a Writer that turns its notes into a cited report.
What you'll learn
- Build RAG pipelines with chunking, embeddings, vector indexes and grounded, cited generation
- Measure and improve retrieval with recall@k and MRR, hybrid search, re-ranking and metadata filters
- Build tool-using agents with validated inputs, retries, approval gates and loop budgets
- Give agents short-term and long-term memory, and plans they validate and repair
- Design multi-agent systems with typed hand-offs, orchestration and peer review
- Diagnose RAG and agent failures from traces and evaluation results
Skills you'll gain
Retrieval-augmented generation, Chunking and indexing, Hybrid search, Re-ranking, Retrieval evaluation, Tool calling, Agent loops, Agent memory, Planning and reflection, Multi-agent orchestration
Tools
Python 3, Anthropic Python SDK, scikit-learn, NumPy, Pydantic, SQLite, sentence-transformers
Before you start
Engineering with Foundation Models, and Prompting and LLM Application Design, whose RAG and ReAct basics this course builds on; Vector Databases and Semantic Search helps with module 1. Install Python 3.10+ and pip install numpy scikit-learn pydantic anthropic (sentence-transformers is optional and downloads small models). Labs run without an API key; their live steps and the capstone need a Claude Console API key with credit and CLAUDE_MODEL set to a current model ID.
Modules
RAG Fundamentals
How retrieval-augmented generation grounds a model in a knowledge base: the indexing and query pipelines, embeddings and vector search, exact and approximate indexes, and why answer quality depends on retrieval quality.
- The RAG architecture: indexing and query time (12 min)
- Embeddings and semantic search (12 min)
- Vector databases and approximate search (12 min)
- Retrieval quality limits answer quality (12 min)
- Hands-on lab: Build a RAG pipeline and measure its retrieval
Advanced Retrieval Techniques
Make retrieval better, and prove it: chunking strategies, BM25 and hybrid search with reciprocal rank fusion, re-ranking with cross-encoders or LLMs, metadata filters and query transformations.
- Chunking strategies (12 min)
- Hybrid search: keywords plus vectors (12 min)
- Re-ranking (12 min)
- Metadata filters and query transformation (12 min)
- Hands-on lab: Chunking, hybrid search, re-ranking and filters
Agent Architecture and Tool Use
From answering to acting: what makes a system an agent, how to design tools a model can use well, the agent loop in code, and the error handling that keeps agents reliable.
- From workflows to agents (12 min)
- Function calling and tool design (12 min)
- The agent loop in code (12 min)
- Error handling and reliability (14 min)
- Hands-on lab: An agent loop that survives failures
Planning and Memory in Agents
Give agents memory within a session and across sessions, plans they can check and run in the right order, and the reflection that lets them recover from mistakes.
- Short-term memory: managing the context window (12 min)
- Long-term memory across sessions (12 min)
- Planning: from goal to steps (12 min)
- Reflection and self-correction (12 min)
- Hands-on lab: Plans, memory and a context budget
Multi-Agent Systems
Teams of specialised agents: when several agents beat one, how they hand work to each other, how to orchestrate them, and how debate and peer review catch mistakes before users see them.
- When to use several agents (12 min)
- Communication protocols and hand-offs (12 min)
- Orchestration frameworks (12 min)
- Debate, peer review and failure modes (12 min)
- Hands-on lab: A researcher, a writer and a reviewer
Project: Autonomous research assistant
Build a multi-agent research assistant that answers this question for Lanternfield, a fictional city: 'What should Lanternfield do next to reduce summer heat in its city centre, and what has worked so far?' A Researcher agent must use tool calling on the Claude API, in a loop you write or the SDK's tool runner, with the two tools in tools.py: search_reports, retrieval over eight local reports (BM25 to start; replace it with embeddings in a vector store such as Chroma or pgvector if you like), and web_search, a stand-in for a search API over a snapshot of web pages, one of them about another town. The Researcher returns structured notes, each with its source ID and the text it relies on. A Writer agent then synthesises a report of 400 to 600 words with a citation after every claim, and a Reviewer, in code, with a model or both, checks each sentence against the notes and sends the draft back until it passes or a revision limit is reached. Keep each model call in its own function, log every tool call and hand-off, set budgets for turns, tool calls and revisions, and handle tool errors. Then evaluate: recall@3 on the retrieval checks in eval_points.json, how many of its seven required points the report covers, citation precision (the share of cited sentences whose source really supports them, checked by you), and any facts borrowed from the other town. Finally, change one thing, such as the retriever, the Researcher's prompt or the review rules, and compare the results.
Part of
The products this course names are trademarks of their owners, who haven't made or endorsed it.