System Design Fundamentals: Scale, Caching and Load Balancing
Design systems that scale: estimate the load, then choose load balancers, caches, databases and queues, and defend every trade-off.
Intermediate · Computer Science · about 17 hours · 5 modules · certificate on completion
Taught by Chaiyapòn Srisomboòn, Arjùn Sharmà, AI Learning Companions.
About this course
System design is the craft of turning a product idea into services, stores and queues that stay fast and correct as traffic grows. This course teaches it the way experienced engineers practise it: pin down the requirements, estimate the load with round numbers, sketch the simplest design that meets them, then go deep on the parts that will break first. You'll learn how load balancers spread traffic and how rate limits, timeouts and circuit breakers stop one failure becoming an outage; how caches and CDNs keep reads fast, and how they go stale; how replication and sharding scale a database; and how queues and idempotency keys make asynchronous work and retries safe. Worked designs of a URL shortener, a chat service and a news feed show the pieces working together.
The labs make the building blocks concrete: you'll write a capacity calculator, a token-bucket rate limiter, an LRU cache with TTLs, a consistent-hash ring and an idempotent payment API in Python, each with pytest tests and a simulation whose output you can check. The capstone asks you to design a live delivery-tracking service and prototype its core from those pieces. Afterwards, Distributed Systems: Replication, Consensus and Consistency goes deeper into replicated stores, Database Internals: Indexes, Transactions and Query Plans goes inside a single database, and Foundations of AI System Design applies the same thinking to machine learning systems.
What you'll learn
- Turn an open-ended brief into measurable requirements, an API and back-of-the-envelope estimates
- Reason about latency with orders of magnitude, percentiles and Little's law
- Choose load-balancing, rate-limiting, caching and CDN strategies, and explain how each one fails
- Scale a database with indexes, read replicas and shards placed by consistent hashing
- Make retries and redeliveries safe with idempotency keys, deduplicating consumers and the outbox pattern
- Build and test a token-bucket limiter, an LRU cache and a consistent-hash ring in Python
Skills you'll gain
System design, Capacity estimation, Load balancing, Caching strategies, Database replication and sharding, Consistent hashing, Rate limiting, Message queues, Idempotent APIs, Trade-off analysis
Tools
Python, pytest, SQLite, Redis, curl
Before you start
Confident Python (functions, classes, dictionaries) and basic SQL: Python Programming with PyCharm and SQL for Data Analysis with DataGrip cover them, and Networking Fundamentals: How Networks and the Internet Work helps with HTTP and DNS. Install Python 3.10 or newer and pytest (python -m pip install pytest). Redis and curl are needed only for three walkthroughs.
Modules
Requirements, estimates and latency
Start every design the same way: pin down what the system must do and how well, estimate its traffic, storage and bandwidth with round numbers, and know each building block's latency to within an order of magnitude.
- From a vague brief to requirements (12 min)
- Back-of-the-envelope estimates (13 min)
- Latency numbers and tail latency (13 min)
- Hands-on lab: Estimate capacity for a photo-sharing service
Load balancing, rate limiting and resilience
Scale out behind load balancers, protect services with rate limits, and keep one slow dependency from taking everything down with timeouts, careful retries and circuit breakers.
- Scaling out: stateless services and load balancers (13 min)
- Timeouts, retries and circuit breakers (12 min)
- Rate limiting algorithms (14 min)
- Hands-on lab: Build a token-bucket rate limiter
Caching and CDNs
Keep hot data in memory close to where it's needed: choose where caches sit and how they're filled, keep them fresh enough, evict sensibly when they're full, and serve cacheable content from the edge.
- Cache-aside and where caches live (12 min)
- Write strategies, invalidation and stampedes (14 min)
- Eviction: LRU, LFU and TTL (12 min)
- CDNs and HTTP caching (13 min)
- Hands-on lab: Build an LRU cache and measure cache-aside
Databases: replication, sharding and consistent hashing
Choose a data store from the access patterns, then scale it: indexes for fast reads, replicas for read capacity and failover, shards for data and write volume, and consistent hashing to place keys. A worked URL shortener puts it together.
- SQL or NoSQL, chosen by access pattern (13 min)
- Replication and read replicas (13 min)
- Sharding and consistent hashing (14 min)
- Worked design: a URL shortener (14 min)
- Hands-on lab: Route short links to shards with a consistent-hash ring
Queues, idempotency and worked designs
Move slow work off the request path with queues, make retries and redeliveries safe with idempotency keys and the outbox pattern, and combine everything in two worked designs: a chat service and a news feed.
- Message queues and asynchronous processing (13 min)
- Idempotency and safe retries (12 min)
- Worked design: a chat service (14 min)
- Worked design: a news feed (14 min)
- Hands-on lab: Make a payment API safe to retry
Project: Design and prototype live delivery tracking
A food-delivery company wants customers to watch their courier approach on a map, updated every few seconds. At the evening peak 50,000 couriers are on shift, each phone sending a GPS update every 4 seconds, and 200,000 customers have a tracking page open. A position older than 30 seconds is useless to a customer, but the company keeps 30 days of history for disputes and route analysis. Write a design document from design_template.md: requirements as numbers, estimates, the API, the data model, a high-level design, and deep dives on ingestion (rate limits, duplicates, out-of-order updates), the latest-position cache, sharding and delivery to viewers, with failure modes and trade-offs. Then prototype the ingestion path in Python from your lab code. make_updates.py writes ten minutes of realistic updates from 200 couriers, with retried duplicates, delayed updates and one misconfigured phone sending every half second; it prints 31,729 rows, 679 duplicates and 228 late updates. Process the rows in arrival order: rate-limit each courier with your token bucket, drop duplicates and stale updates by their per-courier sequence numbers, keep each courier's latest position in your LRU cache with a TTL, and assign couriers to 8 shards with your hash ring. Print the accepted, rate-limited, duplicate and stale counts and the couriers per shard, and test the rules with pytest.
Part of
The products this course names are trademarks of their owners, who haven't made or endorsed it.