A Hands-On Guide to Using SHAP with XGBoost

Q: Why do my SHAP values not sum to the prediction?

Check for link functions. If you used `objective='binary:logistic'`, the raw SHAP values are in log-odds space. Pass `model_output='probability'` to `TreeExplainer` to get probability-space values.

SHAP (SHapley Additive exPlanations) answers a fundamental question: how much did each feature contribute to a specific prediction, relative to the average prediction? For XGBoost models, shap.TreeExplainer computes exact Shapley values efficiently by exploiting the tree structure, giving you consistent, locally accurate attributions that sum to the model output. The core relationship is:

$$\text{prediction} = \text{base_value} + \sum_{i=1}^{M} \text{shap_value}_i$$

where the base value is the expected model output over the background dataset, and each SHAP value represents the feature’s marginal contribution averaged across all possible feature orderings — a concept drawn directly from cooperative game theory’s Shapley values, as formalized for ML by Lundberg & Lee (2017).

Shapley Values in Machine Learning: The Intuition

In cooperative game theory, the Shapley value fairly distributes a coalition’s total payout among players based on their marginal contributions. For ML predictions, features are the players, and the prediction is the payout. The SHAP framework adapts this by computing, for each feature, the average change in prediction when that feature is added to every possible subset of other features.

This yields several desirable properties:

Efficiency: The sum of all SHAP values plus the base value equals the model’s exact prediction. No residual, no approximation.
Symmetry: Features that contribute identically across all coalitions receive identical SHAP values.
Dummy: A feature that never changes the prediction gets zero attribution.
Additivity: For ensemble models like XGBoost, SHAP values decompose naturally across trees.

Critically, SHAP is consistent: if you change your model so a feature becomes more important, its attributed importance never decreases. This is not true for XGBoost’s built-in importance measures (gain, weight, cover), which can produce contradictory rankings across different models trained on the same data.

Setup: TreeExplainer for XGBoost

The TreeExplainer class is optimized specifically for tree-based models. It computes exact SHAP values in polynomial time by pushing subsets of features down the tree structure, avoiding the exponential complexity of naive Shapley computation. For XGBoost, this is dramatically faster than the model-agnostic KernelExplainer.

import xgboost as xgb
import shap
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.datasets import fetch_california_housing

# Load data
data = fetch_california_housing()
X = pd.DataFrame(data.data, columns=data.feature_names)
y = data.target

# Train XGBoost
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)
model = xgb.XGBRegressor(n_estimators=200, max_depth=5, learning_rate=0.1, random_state=42)
model.fit(X_train, y_train)

# Create TreeExplainer
explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X_test)

print(f"Base value (expected prediction): {explainer.expected_value:.4f}")
print(f"SHAP values shape: {shap_values.shape}")  # (n_samples, n_features)

For classification models, shap_values has shape (n_samples, n_features, n_classes) when using pred_contribs=True semantics. Index the appropriate class for interpretation: shap_values[:, :, 1] for the positive class in binary classification.

The newer unified API works identically and is recommended for forward compatibility:

explainer = shap.Explainer(model)
shap_values = explainer(X_test)
# shap_values.values contains the array; shap_values.base_values contains the base values

Visualization Methods

1. Waterfall Plot: Deconstructing a Single Prediction

The waterfall plot shows exactly how each feature pushed the prediction from the base value to the final output for one instance. Features are ordered by impact magnitude.

shap.waterfall_plot(
    shap.Explanation(
        values=shap_values[0],
        base_values=explainer.expected_value,
        data=X_test.iloc[0].values,
        feature_names=X_test.columns.tolist()
    )
)

Red bars indicate features that increased the prediction; blue bars indicate features that decreased it. The values on the right show the cumulative prediction after each feature’s contribution is added. This is the single most intuitive local explanation: you see exactly why this house got that price prediction.

2. Summary Plot: Global Feature Importance with Direction

The summary plot combines feature importance (by mean absolute SHAP value) with the direction of effect. Each dot is one instance’s SHAP value for that feature, colored by the feature’s actual value.

shap.summary_plot(shap_values, X_test)

The color gradient reveals the relationship: if high feature values (red) consistently appear on the positive SHAP side, the feature has a positive monotonic relationship with the target. If red dots appear on both sides, the relationship is non-monotonic or involves interactions. The vertical spread at each feature value shows interaction effects — wider spread means the feature’s impact depends more on other features.

3. Dependence Plot: Feature-Level Detail

To inspect how a specific feature’s SHAP value varies with its actual value:

shap.dependence_plot("MedInc", shap_values, X_test)

Each point represents one instance. The x-axis is the feature value; the y-axis is the SHAP value. A clear upward trend indicates positive correlation with the target. To reveal interactions, color by another feature:

shap.dependence_plot("MedInc", shap_values, X_test, interaction_index="AveRooms")

When the vertical spread changes with the secondary feature’s value, you’re seeing an interaction effect. The SHAP documentation includes extensive examples of dependence plot interpretation.

4. Force Plot: Interactive Local Explanation

The force plot provides a compact, interactive view of a single prediction:

shap.force_plot(explainer.expected_value, shap_values[0], X_test.iloc[0])

Features pushing the prediction higher appear in red on the right; those pushing lower appear in blue on the left. The base value is shown at the boundary. Stacking multiple force plots rotates the view for comparison across instances.

5. Bar Plot: Simple Global Importance

For a clean ranking of overall feature importance:

shap.plots.bar(shap.Explanation(values=shap_values, 
                                base_values=explainer.expected_value, 
                                data=X_test.values,
                                feature_names=X_test.columns.tolist()))

This shows mean absolute SHAP value per feature, which is the most robust global importance metric available.

SHAP Interaction Values

Pairwise feature interactions can be isolated by computing Shapley interaction indices:

explainer_interaction = shap.TreeExplainer(model, feature_dependence="interaction")
shap_interaction_values = explainer_interaction.shap_interaction_values(X_test)

# Shape: (n_samples, n_features, n_features)
# Diagonal elements are main effects; off-diagonal are interaction effects

To visualize a specific interaction:

shap.dependence_plot(
    ("MedInc", "AveRooms"), 
    shap_interaction_values, 
    X_test
)

The plot shows how the MedInc-AveRooms interaction effect varies with MedInc’s value, with points colored by AveRooms. This is invaluable when you suspect features don’t act independently — for instance, income’s effect on housing price may depend on the number of rooms.

SHAP vs. XGBoost Built-in Importance

XGBoost provides three importance metrics: gain (average improvement in accuracy from splits on a feature), weight (number of times a feature is used for splitting), and cover (average coverage of splits). All three have a critical flaw: they lack consistency. A feature can be more important in one model and less important in another, even when the second model simply amplifies that feature’s effect.

SHAP values, being Shapley values, are provably consistent. If feature A has a larger effect on predictions than feature B in every possible context, A will receive higher mean absolute SHAP values. This property holds regardless of model architecture changes, making SHAP the preferred metric for scientific inference and regulatory documentation.

In practice, you’ll often see rough agreement between gain and SHAP importance rankings, but the disagreements are where SHAP earns its place: gain can be misled by features with many low-quality splits, while SHAP reflects actual prediction impact.

Practical Tips for Production Workflows

Sampling for large datasets: Computing SHAP values is $O(TLD^2)$ for $T$ trees, $L$ leaves, and $D$ depth. For datasets with hundreds of thousands of rows, sample for exploratory analysis:

sample_idx = np.random.choice(len(X_test), size=1000, replace=False)
shap_values_sample = explainer.shap_values(X_test.iloc[sample_idx])

Multi-class classification: shap_values will have shape (n_samples, n_features, n_classes). Always index the class of interest explicitly to avoid silently using the wrong class:

shap_values_class_1 = shap_values[:, :, 1]

Persistence: SHAP explainers can be pickled alongside models for reproducibility. However, verify version compatibility between XGBoost and SHAP if deploying across environments.

Background dataset: TreeExplainer uses the training data’s internal node coverages as the background distribution. If you need a different reference population (e.g., a specific demographic subset), pass it via shap.maskers in the new API.

FAQ

Why do my SHAP values not sum to the prediction? Check for link functions. If you used objective='binary:logistic', the raw SHAP values are in log-odds space. Pass model_output='probability' to TreeExplainer to get probability-space values.

Are SHAP values causal? No. SHAP measures predictive association within the model’s learned function. Confounding, collider bias, and other causal structures are not accounted for. For causal inference, use dedicated methods.

Can I use SHAP with custom objective functions? Yes, as long as the model is a valid XGBoost booster. SHAP computes attributions over the learned tree structure, which exists independently of the training objective.

How do I handle categorical features? Pre-process them before training (one-hot, ordinal, or target encoding). SHAP will treat each resulting column as a separate feature. For native XGBoost categorical support, SHAP integrates correctly as of recent versions — consult the SHAP release notes for version-specific details.