Explainable AI: Interpreting Models with SHAP and Beyond

Explain machine learning models in Python: permutation importance, partial dependence, SHAP, local surrogates and counterfactuals, with the checks that show when an explanation can be trusted.

Intermediate · Responsible AI · about 13 hours · 5 modules · certificate on completion

Taught by Deshawné Thompsòn, Amahlé Ndlovù, AI Learning Companions.

About this course

When a model refuses a loan, flags a transaction or ranks a candidate, someone will ask why, and the answer has to be right. This course teaches the main methods for explaining machine learning models and, just as important, how to tell when an explanation can be trusted. You'll explain a gradient boosting model of 5,000 fictional loan applications generated from a known rule, so you can check every explanation against the truth, including the features that only look important because they stand in for others.

You'll start from interpretable models, then measure global behaviour with permutation importance, partial dependence and ICE curves, explain single decisions and whole datasets with SHAP, build a LIME-style local surrogate by hand, and search for counterfactuals an applicant could act on. You'll also explain a text classifier, uncovering a shortcut, and an image classifier, and test the methods with sanity checks. Where AI Ethics in Practice introduces explainability as a principle, this course is the hands-on, technical next step. Every lab prints real results that your instructor checks, and the capstone is an explanation report on a credit model for its reviewers.

What you'll learn

• Choose between interpretable models and post-hoc explanations for a use and an audience
• Measure global feature effects with permutation importance, partial dependence and ICE curves
• Compute and read SHAP values for single predictions and whole datasets, and check their local accuracy
• Build a local surrogate explanation and find actionable counterfactuals for a declined applicant
• Explain text and image classifiers and test explanation methods with sanity checks
• Audit explanations for instability, disagreement, correlated features and misleading references

Skills you'll gain

Explainable AI, Interpretable models, Permutation importance, Partial dependence and ICE, SHAP values, Local surrogate models (LIME), Counterfactual explanations, Explaining text and image models, Auditing explanations

Tools

Python 3, scikit-learn, SHAP, pandas, matplotlib, JupyterLab

Before you start

Machine Learning Foundations with Python, or experience training and evaluating scikit-learn models. AI Ethics in Practice: Fairness, Transparency and Accountability gives useful context but isn't required. Install Python 3.10 or later, then pip install pandas scikit-learn matplotlib shap (the labs use shap's Explainer interface and were run with shap 0.54); JupyterLab is optional.

Modules

1. Why and how models are explained

   The kinds of explanation and who needs them, what interpretable models can show directly, and where even their explanations mislead.

   • What explanations are for, and the kinds there are (12 min)
   • Interpretable models and their limits (12 min)
   • Hands-on lab: Fit glass-box and black-box models

2. Global explanations: permutation importance, PDP and ICE

   Measure which features a model relies on, and how its predictions change with them, using scikit-learn's inspection module, and learn where these methods mislead.

   • Permutation importance (12 min)
   • Partial dependence and ICE curves (12 min)
   • Hands-on lab: Measure global importance and effects

3. SHAP values

   Understand Shapley values and what SHAP computes, use the shap library's Explainer and plots to explain single predictions and whole datasets, and read the results with care.

   • Shapley values: the idea (12 min)
   • SHAP in practice: explainers and plots (12 min)
   • Reading SHAP values carefully (10 min)
   • Hands-on lab: Explain the loan model with SHAP

4. Local explanations: LIME and counterfactuals

   Explain single decisions in two more ways: local surrogate models in the style of LIME, and counterfactuals that say what would have to change, with the constraints that make them useful.

   • LIME and local surrogate models (12 min)
   • Counterfactual explanations and recourse (12 min)
   • Hands-on lab: Build a local surrogate and find counterfactuals

5. Text and image models, and the limits of explanations

   Explain a text classifier and an image classifier, test an explanation method with a sanity check, and bring together what explanations can't do: disagreement, manipulation and over-trust.

   • Explaining text and image models (12 min)
   • The limits of explanations (12 min)
   • Hands-on lab: Explain text and image models, and test the methods

Project: An explanation report for a credit model

A lender's model-risk team must approve the gradient boosting model from this course before it's used, and has asked for an explanation report. Using the course data and model, explain how the model behaves globally and why it decided as it did for five applicants (three declined, two approved), give each declined applicant actionable recourse, and audit the explanations themselves: how stable they are, whether the methods agree, how the correlated pairs (income and deposits, age and credit history) are handled, and which reference the SHAP values use. Write the reasons a declined applicant would receive in plain language, and finish with the 'Explanations and their limits' section of the model's documentation.

Part of

• Data Scientist Professional Certificate

The products this course names are trademarks of their owners, who haven't made or endorsed it.

Start the course free