XGBoost Model Interpretation · SHAP, LIME, and Partial Dependence

XGBoost models achieve high accuracy at the cost of interpretability — a single model with 100 trees and depth 6 is effectively a black box with thousands of decision paths. Model interpretation tools recover explainability without sacrificing performance.

SHAP: The Gold Standard

SHAP (SHapley Additive exPlanations) is the most theoretically grounded interpretation method for tree-based models.

import shap
explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X)

# Global: feature importance with direction
shap.summary_plot(shap_values, X)

# Local: single prediction breakdown
shap.waterfall_plot(explainer(X.iloc[0]))

# Feature interaction
shap.dependence_plot('feature_name', shap_values, X)

Waterfall plot decomposes a single prediction: starting from the expected value, each feature pushes the prediction up or down by its SHAP value magnitude. Red = positive contribution, blue = negative.

LIME: Model-Agnostic Alternative

LIME (Local Interpretable Model-agnostic Explanations) fits a simple linear model locally around the prediction:

from lime.lime_tabular import LimeTabularExplainer
explainer = LimeTabularExplainer(X_train, feature_names=features)
exp = explainer.explain_instance(X_test[0], model.predict)
exp.show_in_notebook()

LIME is slower than SHAP for tree models but works with any model type. Use LIME when SHAP’s TreeExplainer is not available (e.g., for custom ensemble wrappers).

Partial Dependence Plots (PDP)

xgboost-org 配图

Show the marginal effect of a feature on predictions:

from sklearn.inspection import PartialDependenceDisplay
PartialDependenceDisplay.from_estimator(model, X, ['feature_name'])

PDP assumes feature independence — if features are correlated, PDPs can show unrealistic combinations (e.g., 10-bedroom house with 500 sq ft). ICE (Individual Conditional Expectation) plots mitigate this by showing per-instance curves.

Practical Workflow

xgboost-org 配图

  1. Global understanding: shap.summary_plot → identify top features and direction
  2. Feature investigation: shap.dependence_plot for top 3–5 features → detect interactions
  3. Prediction-level debugging: shap.waterfall_plot for misclassified instances → understand why the model was wrong
  4. Regulatory compliance: Document SHAP-based feature importance for model risk management