Q-Learning Explainer
Q-learning is a reinforcement-learning algorithm that estimates an action-value function. A Q-value represents the expected discounted return from taking a particular action in a particular state and then following the best available policy afterward.
After an agent acts, observes a reward, and reaches a new state, it updates the old Q-value toward the immediate reward plus the best estimated future value. A learning rate controls how much the new experience changes the estimate, and a discount factor controls the weight of future rewards.
The agent must balance exploration and exploitation. Always choosing the current best action may prevent discovery of better choices; exploring too much can waste time or cause harm. In small discrete problems, values can be stored in a table. Larger problems often approximate them with functions such as neural networks.
The classic convergence result assumes conditions that real environments may not satisfy, including repeated sampling and a stationary Markov setting. Poor rewards can produce unwanted behavior, and offline or safety-critical exploration needs special controls. Successful simulation results do not by themselves justify unconstrained real-world action.
Q-learning estimates the long-term value of taking an action in a state and then following the best learned choices.
Key terms include state, action, reward, Q-value, policy, discount factor, learning rate, exploration and Bellman update.
Each experience supplies a target built from the immediate reward and the highest estimated value available in the next state.
The stored estimate moves partway toward that target, and repeated experiences gradually reshape the policy implied by the values.
Exploration is needed to learn about alternatives, but unsafe or costly environments limit which experiments are acceptable.
The reward is a proxy for the desired outcome, so omissions or loopholes can lead the agent to optimize behavior the designer did not intend.
Explore more "Explainers"
Discover additional explainers across politics, science, business, technology, and other fields. Each explainer breaks down a complex idea into clear, everyday language—helping you better understand how major concepts, systems, and debates shape the world around us.