Multi-Armed Bandit Explainer
A multi-armed bandit is a simplified sequential decision problem. At each round, a learner chooses one of several actions, observes the reward from that choice, and receives no reward information about the actions it did not select.
The central challenge is exploration versus exploitation. Exploitation chooses the action with the highest estimated reward, while exploration tries uncertain alternatives that may prove better. Common strategies include epsilon-greedy selection, upper confidence bounds, and Thompson sampling.
Bandits are used for experiments, recommendations, advertising, routing, and allocation problems where feedback arrives after each choice. Contextual bandits also use information about the current user or situation before selecting an action.
Evaluation often focuses on cumulative regret, the reward lost relative to a strong reference policy. Real deployments must also address delayed feedback, nonstationary rewards, safety constraints, biased exposure, and the cost of exploration.
A multi-armed bandit models repeated choices where a learner must balance trying uncertain options with using the best-known option.
Key terms include arm, action, reward, exploration, exploitation, epsilon-greedy, upper confidence bound, Thompson sampling and regret.
Exploration gathers information about uncertain actions, while exploitation uses the action currently believed to be best.
A successful strategy adapts that balance as evidence accumulates rather than choosing either extreme permanently.
Rewards may arrive late, change over time, or be observed only for people exposed to a particular action.
Constraints and monitoring are essential when experimental choices can affect safety, fairness, cost, or user trust.
Explore more "Explainers"
Discover additional explainers across politics, science, business, technology, and other fields. Each explainer breaks down a complex idea into clear, everyday language—helping you better understand how major concepts, systems, and debates shape the world around us.
