Model interpretability (SHAP, LIME) in ML Python

Introduction

Model interpretability helps us understand why a machine learning model makes certain decisions. It makes the model's predictions clear and trustworthy.

When you want to explain to a doctor why a model predicts a disease for a patient.

When a bank needs to justify why it approved or denied a loan application.

When you want to check if your model is fair and not biased against certain groups.

When debugging a model to see which features affect predictions the most.

When presenting results to people who do not know machine learning.

Syntax

ML Python

import shap
import lime
from lime import lime_tabular

# For SHAP
explainer = shap.Explainer(model, data)
shap_values = explainer(data_to_explain)

# For LIME
explainer = lime_tabular.LimeTabularExplainer(training_data, feature_names=features, class_names=classes, mode='classification')
exp = explainer.explain_instance(instance_to_explain, model.predict_proba)

SHAP explains model predictions by assigning each feature an importance value called a SHAP value.

LIME explains predictions by approximating the model locally with a simple interpretable model.

Examples

This example shows how to create a SHAP explainer and plot the explanation for one test example.

ML Python

import shap
explainer = shap.Explainer(model, X_train)
shap_values = explainer(X_test[0:1])
shap.plots.waterfall(shap_values[0])

This example shows how to use LIME to explain one prediction and display it in a notebook.

ML Python

from lime import lime_tabular
explainer = lime_tabular.LimeTabularExplainer(X_train, feature_names=feature_names, class_names=class_names, mode='classification')
exp = explainer.explain_instance(X_test[0], model.predict_proba)
exp.show_in_notebook()

Sample Model

This program trains a random forest on the iris dataset. Then it uses SHAP and LIME to explain the prediction for the first test sample. It prints the importance of each feature according to both methods.

ML Python

import numpy as np
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
import shap
from lime import lime_tabular

# Load data
iris = load_iris()
X, y = iris.data, iris.target
feature_names = iris.feature_names
class_names = iris.target_names

# Split data
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)

# Train model
model = RandomForestClassifier(random_state=42)
model.fit(X_train, y_train)

# SHAP explanation
explainer_shap = shap.Explainer(model, X_train)
shap_values = explainer_shap(X_test[0:1])

# Print SHAP values for first test sample
print('SHAP values for first test sample:')
for name, val in zip(feature_names, shap_values.values[0]):
    print(f'{name}: {val:.4f}')

# LIME explanation
explainer_lime = lime_tabular.LimeTabularExplainer(X_train, feature_names=feature_names, class_names=class_names, mode='classification')
exp = explainer_lime.explain_instance(X_test[0], model.predict_proba)
print('\nLIME explanation for first test sample:')
for feature, weight in exp.as_list():
    print(f'{feature}: {weight:.4f}')

Important Notes

SHAP values add up to the difference between the model prediction and the average prediction, making them easy to interpret.

LIME creates a simple model around one prediction, so explanations can differ for each instance.

Both methods help build trust by showing which features influence decisions.

Summary

Model interpretability helps explain why models make certain predictions.

SHAP and LIME are popular tools that show feature importance for individual predictions.

Using these tools makes machine learning models more transparent and trustworthy.