Introduction
A good model in a notebook is not enough. Stakeholders need to understand why the model makes the decisions it does, and the organization also needs to know if the model is biased. This is where explainability and fairness become imperative.
Lesson objectives
- Understand the difference between local and global explainability.
- Know how to use permutation importance or SHAP at an introductory level.
- Know how to ask fairness questions before deploying.
Explainability to whom?
- Data scientists need to debug the model.
- PM needs to understand which features affect results.
- Professional stakeholders need simple, trustworthy stories.
- End users may need specific reasons for an individual prediction.
Two levels of explanation
- Global explainability: what factors does the model generally rely on?
- Local explainability: why a particular sample is predicted like that.
Fairness is not a secondary option
If the model influences decisions with humans, you should test the performance for each user group, the rate of false positives or false negatives by each group, and which features can be proxies for sensitive attributes.
How to present to stakeholders
- Avoid lengthy formulas.
- Use 3 to 5 most influential features.
- Clearly state the confidence level and limits of the model.
- State the risks of bias frankly.
Common mistakes
- Abusing the SHAP chart without understanding its nature.
- Confusing explainability with proving cause and effect.
- Only check fairness after deployment.
Practice exercises
- Choose a trained classification model.
- Create permutation importance or SHAP summary.
- Check metrics by 2 different user groups.
- Write the explanation as if you were presenting to the PM.
Completion criteria
- Distinguish between global and local explainability.
- Know at least one basic fairness check.
- Present results that non-technical stakeholders can still understand.
Practice step by step (advanced)
- Choose a trained model and a fixed validation set.
- Calculate global importance and local explanation for 5 samples.
- Divide data into 2-3 user groups to compare fairness.
- Record the metric difference by group.
- Prepare ethical risk memo and control recommendations.
Artifact should be submitted
- 1-2 page explainability report.
- Fairness metrics table by group.
- List of actions to reduce bias by priority.
Self-test questions
- Why does explainability not prove causality?
- How much metric difference is alarming?
- When should I refuse to deploy because of fairness risk?