Reinforcement Learning from Human Feedback Explainer
Reinforcement learning from human feedback, or RLHF, is a family of methods that uses human judgments to shape model behavior. In language-model training, people may compare several responses to the same prompt and indicate which better follows the desired criteria.
A common pipeline begins with supervised instruction tuning. A separate reward model then learns to predict human preferences from comparison data. The language model is optimized to produce outputs that score well under that learned reward while constraints limit how far it moves from a reference model.
RLHF can improve helpfulness and instruction following, but the reward model is only a proxy for the judgments in its dataset. Labelers may disagree, criteria may be incomplete, and a model can exploit patterns that raise reward without achieving the intended result. Preference data also reflect who was asked and how the task was framed.
The method does not guarantee truth, fairness, or safety. Developers need independent evaluations, red-team tests, monitoring, and ways to handle feedback from affected users. Alternatives such as direct preference optimization change the training procedure, but they still depend on the quality and scope of preference data.
Reinforcement learning from human feedback uses human comparisons to train a model toward preferred behavior.
Key terms include human preference, comparison data, reward model, policy, supervised fine-tuning, optimization, proxy objective, labeler disagreement and alignment.
Human rankings train a reward model to score new outputs, and the language model is then updated to favor responses with higher predicted reward.
Constraints are commonly used to preserve useful behavior from the starting model and reduce unstable changes during optimization.
Feedback reflects the instructions, cultural context, expertise, time, and working conditions of the people who provide it.
A strong reward score can conceal factual errors or reward hacking, so independent task and safety evaluations remain necessary.
Explore more "Explainers"
Discover additional explainers across politics, science, business, technology, and other fields. Each explainer breaks down a complex idea into clear, everyday language—helping you better understand how major concepts, systems, and debates shape the world around us.