What Is RLHF? Reinforcement Learning from Human Feedback

The basic idea

After pretraining, human reviewers rank multiple model outputs for the same prompt from best to worst. Those rankings train a separate 'reward model' that learns to predict which outputs humans prefer.

Closing the loop

The base language model is then fine-tuned using reinforcement learning against that reward model, nudging it to produce more of what humans rated highly and less of what they rated poorly.

Why it was a breakthrough

RLHF is largely what turned raw, next-token-predicting base models into the helpful, instruction-following assistants people interact with today — the difference between a model that completes text and one that actually converses.