Reinforcement Learning from AI Feedback
RLAIF
An AI helps judge the practice.
An AI evaluator supplies feedback. A common setup trains a reward model on its comparisons, then uses that model for reinforcement learning.
The idea
A teacher writes a grading rubric. An automated teaching assistant uses it to compare essays. More essays can be graded, but a mistake in the rubric or grader can affect many students.
The judge is a model following a rubric.
Give useful feedback on a student's draft.
Your example is clear. Add one sentence explaining how it supports your point.
This is bad. Rewrite it.
An AI judge uses the rule “be helpful and respectful” and prefers A.
AI preference labels → train a reward model
Its scores guide the assistant's RL practice.
How RLAIF works
Set the judging rules
People specify the behavior they want.
Get AI feedback
An evaluator compares candidate answers using those rules.
Turn feedback into rewards
For example, train a model to predict the evaluator's preferences.
Train and check
Use RL to improve the assistant, then check its behavior independently.
Useful for
Collecting more preference feedback than people can feasibly label by hand.
The catch
An AI judge can be biased, fooled, or confidently wrong. AI feedback does not remove the need for human choices and evaluation.
A little more detail
Is this the same as Constitutional AI?
Not exactly. Constitutional AI is a broader recipe guided by written principles. Its original version includes both a critique-and-revision stage and an RLAIF stage.
RLAIF names the use of AI feedback; it is not one fixed set of principles.
Read the research
These guides simplify the methods. The papers describe the full training recipes. Examples on this page are illustrative, not experimental results.