Subsampling in XGBoost: A Complete Guide

Subsampling is one of the most powerful and frequently misunderstood regularization techniques in XGBoost. It works by injecting randomness into the tree-building process, forcing the ensemble to rely on multiple weak signals rather than memorizing the training data. The core principle is simple: before constructing each tree (or each level, or each node), we randomly draw a subset of the available data — rows, columns, or both. This controlled randomness reduces overfitting, speeds up training, and often improves generalization performance at no cost to model capacity.

XGBoost provides four distinct subsampling parameters: subsample for rows, and colsample_bytree, colsample_bylevel, and colsample_bynode for columns. They can be combined freely, and their effects compound in ways that are both predictable and tunable. Understanding how they interact is essential for building robust gradient boosting models.

Row Subsampling: The subsample Parameter

Row subsampling is controlled by the subsample parameter. Before building each tree, XGBoost randomly selects a fraction subsample of the training instances without replacement. If subsample=0.8, each tree sees only 80% of the data, and the 20% left out changes from tree to tree.

This idea originates in Friedman’s 2002 paper “Stochastic Gradient Boosting”, which demonstrated that introducing stochasticity into the gradient boosting procedure improves accuracy while reducing computational cost. The insight is twofold:

Variance reduction through ensemble diversity. When each tree is built on a different subset of rows, the trees become less correlated with one another. The final prediction is an average (or sum) of many individually imperfect trees. Because their errors are partially uncorrelated, they cancel out in the aggregate. This is the same principle that makes bagging effective in Random Forests, but applied to the boosting framework.

Computational speedup. Training time scales roughly linearly with the number of instances. With subsample=0.5, each tree trains on half the data, and you get approximately a 2× speedup per boosting round. This can be traded off against n_estimators: you can train twice as many trees in the same wall-clock time if each tree uses half the data.

How It Affects the Learning Dynamics

Subsampling does not modify the objective function directly. Instead, it alters the gradient statistics used during split finding. At each boosting round, XGBoost computes the gradients and Hessians of the loss function with respect to the current predictions. When subsample < 1.0, these gradient statistics are computed only on the sampled subset, so the tree optimizes an approximate, noisier version of the true gradient.

This noise acts as a regularizer. A tree that sees only 60% of the data cannot perfectly fit the residuals — it is forced to learn coarser, more generalizable patterns. The trade-off is that lower subsample values require more boosting rounds to converge, much like lowering the learning rate eta. A common pattern: subsample=0.8 with n_estimators=100 may achieve similar training loss to subsample=0.5 with n_estimators=200, but the latter often generalizes better.

Typical values range from 0.5 to 1.0. The default in the XGBoost library is 1.0 (no subsampling). The XGBoost documentation recommends starting at 1.0 and reducing to 0.8 or 0.6 if overfitting is observed. Values below 0.5 are unusual but can be effective on large, noisy datasets.

The sampling_method Parameter

XGBoost supports two row sampling strategies via the sampling_method parameter:

• uniform (default): Each instance has equal probability of being selected. This is the classic stochastic gradient boosting approach.
• gradient_based: Instances are sampled with probability proportional to the magnitude of their gradient (specifically, the absolute value of the first-order gradient). Instances with larger gradients — those the model is currently fitting poorly — are more likely to be included.

Gradient-based sampling is particularly useful for imbalanced datasets or problems where certain regions of the feature space are harder to fit. By focusing each tree on the “difficult” examples, it can accelerate convergence on the minority class or high-error regions. However, it introduces a bias that must be corrected via importance weighting; XGBoost handles this internally.

The XGBoost parameter documentation provides full details on sampling_method and its interaction with subsample.

Column Subsampling: Feature-Level Regularization

Column subsampling reduces overfitting by restricting the features available at each split decision. This is directly analogous to the feature bagging used in Random Forests, and it serves the same purpose: decorrelating trees by forcing them to solve the problem using different subsets of predictors.

XGBoost offers three levels of column subsampling, which are applied sequentially:

colsample_bytree

A fraction of columns (features) is sampled once per tree. All splits within that tree use the same feature subset. This is the coarsest level of column subsampling and the most commonly tuned.

Typical range: 0.5–1.0. On high-dimensional datasets with many noisy or irrelevant features, values as low as 0.3 can be effective. On low-dimensional data (fewer than, say, 20 features), values below 0.7 risk excluding important predictors too often.

colsample_bylevel

A fraction of columns is sampled once per level (depth) of each tree. Within a given level, all nodes use the same feature subset, but the subset changes as the tree grows deeper. This provides finer-grained regularization than colsample_bytree.

Because different levels of a tree often capture interactions at different scales, colsample_bylevel can be especially useful when feature interactions are complex and multi-scale. However, it is less aggressive than node-level sampling and computationally cheaper.

colsample_bynode

A fraction of columns is sampled at every split node. This is the finest-grained column subsampling and the most aggressive regularizer. Every split decision sees a different random subset of features.

This parameter was introduced to mirror the split-level randomization in libraries like LightGBM. It provides the maximum possible decorrelation between splits but can sometimes be too aggressive, preventing the tree from learning coherent structure.

How the Three Column Parameters Compose

The three column subsampling parameters are applied sequentially. The effective fraction of features considered at a given node is:

$$f_{\text{effective}} = \text{colsample_bytree} \times \text{colsample_bylevel} \times \text{colsample_bynode}$$

For example, if colsample_bytree=0.8, colsample_bylevel=0.8, and colsample_bynode=0.8, then only $0.8^3 = 0.512$ of features are available at any given split. This multiplicative effect means that tuning all three simultaneously can easily lead to overly aggressive subsampling. In practice, most practitioners tune only one or two of these parameters, leaving the others at 1.0.

Combined Effect: Row + Column Subsampling

When both row and column subsampling are used together, the regularization effect is multiplicative and extremely robust. Each tree sees a different subset of rows and a different subset of columns, meaning that no two trees in the ensemble are likely to make the same errors. The ensemble averages out both instance-level noise (via row subsampling) and feature-level noise (via column subsampling).

This combination is one of the reasons gradient boosting with subsampling often outperforms both unregularized boosting and bagged ensembles like Random Forests on structured data. The stochasticity prevents the greedy tree-building algorithm from overcommitting to spurious patterns while still allowing the ensemble as a whole to capture complex interactions.

Interaction with n_estimators and learning_rate

Subsampling, like shrinkage (eta), increases the number of boosting rounds required for convergence. The relationship is approximately:

$$\text{required trees} \propto \frac{1}{\text{subsample} \times \text{colsample_bytree}}$$

If you halve subsample from 1.0 to 0.5, you should roughly double n_estimators to maintain the same training loss. This interacts with early stopping: aggressive subsampling combined with early stopping can terminate training before the model has converged, leading to underfitting. Always monitor validation metrics when tuning subsampling parameters.

Practical Tuning Strategy

A systematic approach to subsampling tuning:

1. Start with no subsampling: subsample=1.0, all colsample_* parameters at 1.0. Establish a baseline with an appropriate learning_rate and n_estimators (using early stopping).

2. If overfitting (validation error increasing while training error decreases): Reduce subsample to 0.8. If overfitting persists, try 0.6. Row subsampling is usually the first and most impactful subsampling parameter to tune.

3. For high-dimensional data (hundreds or thousands of features): Reduce colsample_bytree to 0.5–0.8. If features are highly correlated or noisy, consider more aggressive values (0.3–0.5). Start with colsample_bytree alone before introducing colsample_bylevel or colsample_bynode.

4. Fine-tune column subsampling granularity: If colsample_bytree helps but more regularization is needed, introduce colsample_bylevel or colsample_bynode at conservative values (0.8–0.9). Avoid using all three at low values simultaneously unless you have a very large feature set and strong evidence of overfitting.

5. Increase n_estimators to compensate for the stochasticity. With subsample=0.6 and colsample_bytree=0.5, you may need 3–4× more trees than the baseline. Use early stopping to find the optimal point.

6. Experiment with sampling_method='gradient_based' on imbalanced datasets. This can provide additional benefit beyond uniform subsampling by focusing computation on hard examples.

FAQ

Does subsampling replace the need for eta (learning rate)? No. Shrinkage and subsampling are complementary regularizers. Shrinkage reduces the contribution of each tree; subsampling reduces the correlation between trees. They are often used together, and both increase the required number of trees.

Can I use subsample=0.5 with colsample_bytree=0.5? Yes, and this combination can be very effective on large, noisy datasets. However, expect to need significantly more boosting rounds. The effective data seen by each tree is only 25% of the original (50% of rows × 50% of columns), so plan for 4× or more trees compared to the unsubsampled baseline.

What happens if my dataset is small? On small datasets (fewer than ~1,000 instances), aggressive row subsampling can lead to high variance in the gradient estimates and poor convergence. Stick to subsample ≥ 0.8 unless you have strong evidence that lower values help. Column subsampling is generally safer on small datasets, especially if the feature count is moderate.

How does subsampling interact with GPU training? Subsampling works identically on GPU and CPU, but the performance gains from row subsampling may be less dramatic on GPU because GPU training is already highly parallelized. The regularization benefits remain the same regardless of hardware. Always verify parameter behavior against the official XGBoost documentation for your specific version.