Math
Derivations behind XGBoost: second-order approximation, gain, gamma pruning and regularisation. 3 articles.
XGBoost Learning Rate · Eta, Shrinkage, and Tree Count
The relationship between XGBoost's learning rate (eta) and number of estimators — the shrinkage principle, optimal ratios, and the diminishing returns of smaller learning rates.
XGBoost Regularization · L1 and L2 in Gradient Boosting
How XGBoost's regularization parameters (lambda, alpha, gamma) control model complexity — the mathematical formulation, practical tuning strategies, and how they differ from ordinary linear model regularization.
How `gamma` Controls Tree Pruning in XGBoost
The gamma hyperparameter explained — how it acts as a complexity penalty per leaf, why higher gamma means simpler trees, and how to tune it for your dataset.