XGBoost Monotonic Constraints · Ensuring Predictions Follow Domain Logic

Monotonic constraints ensure that a model’s predictions follow expected directional relationships — for example, that higher income never leads to lower predicted creditworthiness. XGBoost supports feature-level monotonic constraints natively, which is essential in regulated domains where model behavior must be explainable and compliant.

Problem: Unconstrained Models Violate Domain Logic

An unconstrained XGBoost model predicting loan default probability might learn that “income = $95K” has a higher default risk than “income = $90K” because of noise or confounders in the training data. While this may be “correct” from a correlation standpoint, it violates domain logic and is unacceptable in regulated lending.

Solution: Monotonic Constraints

params = {
    'monotone_constraints': '(1, 0, -1, 0, 1)',
    # Feature 0: increasing (higher → higher prediction)
    # Feature 1: no constraint    
    # Feature 2: decreasing (higher → lower prediction)
    # Feature 3: no constraint
    # Feature 4: increasing
}

Or via scikit-learn API:

model = xgb.XGBRegressor(
    monotone_constraints={'income': 1, 'debt_to_income': -1, 'credit_history_length': 1}
)
  • 1: Monotonically increasing (higher $x$ → higher $\hat{y}$)
  • -1: Monotonically decreasing (higher $x$ → lower $\hat{y}$)
  • 0: No constraint (default)

How It Works

During tree construction, XGBoost checks that candidate splits satisfy all active monotonic constraints. A split that violates any constraint is rejected — the algorithm effectively prunes the search space to constraint-satisfying splits only.

This is enforced at the leaf weight level: the weights of left and right children must satisfy the monotonicity condition for all constrained features. If a feature $j$ has constraint $+1$, then the right child’s weight must be ≥ the left child’s weight when the split is on feature $j$.

Use Cases

xgboost-org 配图

Credit scoring: Income ↑ → default probability ↓, debt-to-income ↑ → default probability ↑. These should be enforced.

Pricing models: Quantity ↑ → unit price ↓ (volume discounts). Square footage ↑ → price ↑ (real estate).

Insurance: Age ↑ → premium ↑ (life insurance, after a certain age). Driving experience ↑ → premium ↓.

Medical risk: Blood pressure ↑ → cardiovascular risk ↑ (within reasonable ranges). Exercise frequency ↑ → risk ↓.

Trade-offs

Benefit: Guaranteed logically coherent predictions. Critical for regulatory compliance and stakeholder trust.

Cost: Potentially reduced predictive accuracy. If the true relationship is non-monotonic (U-shaped, like age vs. insurance risk), the constraint will force a monotonic approximation that may be less accurate.

Guideline: Constrain only features where the monotonic relationship is known with high confidence. Over-constraining leads to systematic bias.

Verification

xgboost-org 配图

# Verify monotonicity after training
import numpy as np
feature_idx = 0
test_vals = np.linspace(X[:, feature_idx].min(), X[:, feature_idx].max(), 100)
preds = []
for val in test_vals:
    X_test = X_median.copy()
    X_test[:, feature_idx] = val
    preds.append(model.predict(X_test))
assert all(preds[i] <= preds[i+1] for i in range(len(preds)-1)), "Non-monotonic!"