Foundations of Probability and Statistics
Build the probability that data science rests on: rules and counting, Bayes' theorem, discrete and continuous distributions, expectation and inference, derived, calculated and checked by simulation.
Intermediate · Data Science · about 14 hours · 5 modules · certificate on completion
Taught by Ferencz Nagŷ, Bintà Yusupovà, AI Learning Companions.
About this course
Every model in data science is built on probability: a classifier's scores, an A/B test's verdict and a forecast's interval all rest on a handful of rules and distributions. This course builds them from the ground up, with the mathematics written out and every result checked in Python. You'll start with sample spaces, the axioms of probability and counting, and learn to tell mutually exclusive events from independent ones. Then you'll update probabilities with evidence: conditional probability, the law of total probability and Bayes' theorem, in its probability and odds forms, applied to a screening test whose positives are mostly false, and to a proportion whose prior becomes a posterior.
Next come random variables and their distributions: PMFs and CDFs, expected values and variances, the binomial, geometric, Poisson and hypergeometric for counts, and the uniform, exponential and normal for measurements, with the approximations that link them. You'll see the law of large numbers and the central limit theorem at work, and where the theorem fails. Finally you'll turn to inference: estimators, bias and maximum likelihood, confidence intervals and their real coverage, and hypothesis tests, p-values, errors and power. Every lab calculates exact answers with SciPy and confirms them by simulation with NumPy, and the capstone is a Monte Carlo study that tests the central limit theorem and finds empirical p-values for a die suspected of favouring six. It prepares you for Practical Statistics for Data Scientists.
What you'll learn
- Calculate probabilities with the axioms, counting, the addition and complement rules, and independence
- Apply conditional probability, the law of total probability and Bayes' theorem to update beliefs with evidence
- Work with discrete and continuous random variables: PMFs, densities, CDFs, expected values and variances
- Choose and use the binomial, geometric, Poisson, uniform, exponential and normal distributions, and their approximations
- Estimate parameters, build confidence intervals and run hypothesis tests, and check them by Monte Carlo simulation
Skills you'll gain
Probability rules, Counting and combinatorics, Conditional probability, Bayes' theorem, Random variables, Expectation and variance, Probability distributions, Central limit theorem, Estimation and confidence intervals, Monte Carlo simulation
Tools
Python 3, NumPy, SciPy, JupyterLab
Before you start
School algebra, and Python with NumPy at the level of Python Foundations for Data Science; each SciPy call is explained as it's used. Statistics for Data Science and AI is a gentler, applied introduction; this course goes deeper into the theory: counting, Bayes, expectation, the distributions and why inference works. Install Python 3 with NumPy and SciPy (1.9 or later); JupyterLab is optional.
Modules
Basic Probability Concepts
Set probability on firm ground: sample spaces and events, the axioms and the rules that follow from them, counting outcomes, and the difference between mutually exclusive and independent events.
- Sample spaces, events and the axioms (12 min)
- Counting outcomes: permutations and combinations (12 min)
- The addition rule, mutually exclusive and independent events (12 min)
- Hands-on lab: Count, calculate and simulate
Conditional Probability and Bayes' Theorem
Update probabilities with evidence: conditional probability and the multiplication rule, the law of total probability, Bayes' theorem in its probability and odds forms, and Bayesian updating of a proportion.
- Conditional probability and the multiplication rule (12 min)
- The law of total probability (10 min)
- Bayes' theorem: from prior to posterior (12 min)
- Bayesian updating of a proportion (12 min)
- Hands-on lab: Bayes in practice: emails, suppliers and screening tests
Discrete Probability Distributions
Describe random quantities with probability mass and cumulative distribution functions, calculate expected values and variances, and use the binomial, geometric, Poisson and hypergeometric distributions.
- Random variables, PMFs and CDFs (10 min)
- Expected value and variance (12 min)
- Bernoulli, binomial, geometric and hypergeometric (12 min)
- The Poisson distribution: events in an interval (12 min)
- Hands-on lab: Discrete distributions: calculate, simulate, compare
Continuous Probability Distributions
Work with continuous random variables through densities and cumulative distributions: the uniform, exponential and normal distributions, standardisation and normal approximations, and the law of large numbers and the central limit theorem.
- Densities, CDFs and quantiles (12 min)
- The normal distribution and standardisation (12 min)
- The law of large numbers and the central limit theorem (12 min)
- Hands-on lab: Continuous distributions and the central limit theorem
Introduction to Statistical Inference
Go from data to conclusions: estimators and their bias and standard error, maximum likelihood, confidence intervals and their coverage, and hypothesis tests, p-values, errors and power, all checked by simulation.
- Estimators: bias, standard error and maximum likelihood (12 min)
- Confidence intervals (12 min)
- Hypothesis tests, p-values and power (12 min)
- Hands-on lab: Estimate, test and check by simulation
Project: Statistical Simulation Lab: the central limit theorem and empirical p-values
Design and run a Monte Carlo study, as a script or a notebook, that answers two questions by simulation and checks each answer against theory. Part 1, the central limit theorem: for at least three populations of different shapes (for example uniform on 0 to 1, exponential with mean 1, and Bernoulli with p = 0.1) and sample sizes of 2, 5, 30 and 100, draw at least 20,000 samples each and keep their means. Compare the simulated standard deviation of the means with σ/√n, their skewness with the population's skewness divided by √n, and the share of means above μ + 1.645σ/√n with the 0.05 the normal approximation predicts; draw histograms with the normal curve on top; and decide for which population and n the approximation becomes good enough, and why skewness matters. Part 2, empirical p-values: die_rolls.txt holds 120 rolls of a die suspected of favouring six (made-up data). State the hypotheses, compute the chi-square goodness-of-fit statistic, and find its p-value by simulating at least 100,000 sets of 120 fair rolls (with the +1 correction), then with scipy.stats.chisquare. Test the count of sixes too, by simulation and with stats.binomtest, and explain why the two tests disagree. Finally, estimate by simulation the power of both tests at the 5% level against a die with P(6) = 0.25, and how many rolls the chi-square test needs for 80% power. Fix your random seed and give each simulated probability with its Monte Carlo standard error, √(p(1 - p)/N). Check yourself: the faces' counts are 16, 18, 20, 16, 21 and 29; the chi-square statistic is 5.90 with SciPy's p-value 0.316; and the one-sided binomial p-value for 29 sixes is 0.022.
Part of
The products this course names are trademarks of their owners, who haven't made or endorsed it.