XGBoost for Regression · Objective Functions and Metrics
XGBoost supports a rich set of regression objectives beyond standard squared error. Choosing the right objective aligns the model with the error structure of your data and the business cost of prediction errors.
Built-in Regression Objectives
reg:squarederror (default)
Minimizes MSE. Optimal when errors are normally distributed. Sensitive to outliers — a single extreme error dominates the gradient.
model = xgb.XGBRegressor(objective='reg:squarederror')reg:absoluteerror (MAE)
Minimizes mean absolute error. More robust to outliers than squared error. Gradient is ±1 regardless of error magnitude. Converges more slowly.
reg:pseudohubererror
Huber loss in XGBoost: squared error for small residuals, absolute for large. Parameter huber_slope controls the transition point.
model = xgb.XGBRegressor(objective='reg:pseudohubererror', huber_slope=1.0)reg:tweedie
Tweedie distribution family, parameterized by tweedie_variance_power:
- 0: Normal (squared error)
- 1: Poisson (count data, mean = variance)
- Between 1 and 2: Compound Poisson–Gamma (insurance claims — zeros + positive continuous)
- 2: Gamma (positive continuous, constant CV)
- 3: Inverse Gaussian
# Insurance claim severity (many zeros, positive claims)
model = xgb.XGBRegressor(objective='reg:tweedie', tweedie_variance_power=1.5)reg:gamma
For strictly positive targets with variance proportional to mean². Useful for claim severity, time-to-event, cost modeling.
reg:quantileerror
Quantile regression. Parameter quantile_alpha (0–1). Predicts the $\alpha$-th quantile. For prediction intervals, train two models at $\alpha = 0.05$ and $\alpha = 0.95$.
Choosing by Data Type
| Data Pattern | Recommended Objective |
|---|---|
| Normally distributed errors | squarederror |
| Heavy-tailed, outliers present | pseudohubererror or absoluteerror |
| Count data | count:poisson |
| Positive, right-skewed (cost, duration) | gamma |
| Zero-inflated positive (claims) | tweedie (1 < p < 2) |
| Prediction intervals needed | quantileerror (α = 0.05 & 0.95) |
Evaluation Metrics
model = xgb.XGBRegressor(
objective='reg:squarederror',
eval_metric=['rmse', 'mae']
)rmse: Root mean squared error (same scale as target)mae: Mean absolute error (robust, same scale)mape: Mean absolute percentage error (scale-independent, undefined for zero targets)rmsle: Root mean squared log error (penalizes under-prediction heavily)