Machine Learning Bagging Explainer
Bagging, short for bootstrap aggregating, trains multiple base models on different resampled versions of the training data. Each bootstrap sample is drawn with replacement, so some records appear more than once and others are left out.
For regression, predictions are usually averaged. For classification, models may vote or average probabilities. Combining unstable estimators such as deep decision trees can reduce variance and improve performance on new data.
Random forests extend bagging by also selecting random feature subsets when building tree splits. The added randomness helps reduce correlation between trees, making the average more effective.
Bagging does not automatically fix bias, weak features, leakage, or unrepresentative data. The number and complexity of estimators affect compute and memory, while out-of-bag examples can provide a useful internal performance estimate under suitable conditions.
Bagging trains multiple versions of a model on bootstrap samples and combines their predictions to reduce variance.
Key terms include bootstrap sample, sampling with replacement, aggregation, base estimator, variance, out-of-bag example, random forest and correlation.
Each base model sees a different bootstrap sample, and the final predictor averages or votes across the resulting models.
The variation among samples creates multiple views of the data instead of one fragile fit.
Averaging reduces variance most effectively when component errors are not strongly correlated.
Feature randomness and other design choices can diversify models, but they should not obscure problems in the underlying data.
Explore more "Explainers"
Discover additional explainers across politics, science, business, technology, and other fields. Each explainer breaks down a complex idea into clear, everyday language—helping you better understand how major concepts, systems, and debates shape the world around us.
