What Is Bayesian Optimization?

Bayesian Optimization Explainer

Bayesian optimization is a strategy for optimizing an expensive function when each evaluation costs substantial time or resources. It builds a probabilistic surrogate model from completed trials instead of exploring every setting directly.

An acquisition function uses the surrogate’s predictions and uncertainty to select the next trial. Some choices favor settings likely to improve the current best result, while others give more weight to uncertain regions that may contain better solutions.

The method is used for hyperparameter tuning, laboratory experiments, engineering design, and other black-box problems with limited evaluation budgets. It is most attractive when fitting the surrogate is cheaper than running the real experiment.

Performance depends on the search space, surrogate assumptions, acquisition strategy, noise, constraints, and parallel evaluations. It can struggle with many dimensions or changing objectives, and it does not guarantee that a small budget will find the global optimum.

Bayesian optimization selects promising experiments by balancing uncertain model predictions against the value of exploring new settings.

Key terms include Bayesian optimization, black-box function, surrogate model, acquisition function, exploration, exploitation, evaluation budget and constraint.

The surrogate summarizes what completed evaluations imply about untried settings.

The acquisition function converts predicted value and uncertainty into a decision about where to evaluate next.

Bayesian optimization saves effort when the real experiment is much costlier than updating the surrogate.

Search-space design and dimensionality can matter more than the sophistication of the acquisition rule.

Explore more "Explainers"

Discover additional explainers across politics, science, business, technology, and other fields. Each explainer breaks down a complex idea into clear, everyday language—helping you better understand how major concepts, systems, and debates shape the world around us.