Reinforcement Learning from Human Feedback (RLHF)
Definition
RLHF is a training method where people rank or rate a model's outputs, and those human preferences are used to train a reward model that then fine-tunes the AI to behave the way people prefer.
A base language model can predict text, but it does not know which of two plausible answers a person would actually find more helpful, honest or safe. RLHF closes that gap. Human raters compare model outputs, the comparisons train a reward model that scores any output, and reinforcement learning optimises the main model against that reward.
The quality of RLHF depends almost entirely on the quality of the human feedback. Raters who do not understand the domain produce noisy or wrong preferences, and the model learns those mistakes. For specialised subjects such as medicine, law or engineering, generic crowd raters are often not enough; the comparison has to be made by someone who knows the correct answer.
RLHF is distinct from supervised fine-tuning (where the model copies example answers) and from newer variants such as direct preference optimisation, which skip the separate reward model but still rely on human preference data.
How Pathwize relates
Pathwize supplies the human preference data for RLHF from credential-checked domain experts rather than generic crowds, with a qualification test on your own data and the rater's identity, notes and time recorded on every comparison.
Frequently asked
People rate a model's answers, those ratings train a reward model, and the AI is tuned to produce answers people prefer.
For specialised topics, only someone who knows the correct answer can judge which output is actually better; a wrong preference teaches the model the wrong thing.
Supervised fine-tuning copies example answers. RLHF learns from comparisons between answers, which captures preferences that are hard to write down as examples.
Related terms