Group Relative Policy Optimization
GRPO
Compare several attempts at the same task.
Generate several answers to one prompt. Compare their rewards within the group. Use those relative results to update the model.
The idea
You try four solutions to the same puzzle. Comparing those tries shows which did better within this practice round. It does not tell you whether the puzzle was a good test in the first place.
One prompt. Several attempts. Relative feedback.
Write a function that passes these tests.
0 tests passed
Reward 02 tests passed
Reward 24 tests passed
Reward 4Toy group average = 2. A is below it; B matches it; C is above it.
Relative advantages → an RL update
Generate a new group with the updated model.
How GRPO works
Sample a group
Ask the current model for several answers to the same prompt.
Score the answers
Use a verifier or another chosen reward source.
Compare within the group
Estimate how each answer performed relative to its peers.
Update and repeat
Adjust the policy using those relative signals and constraints on the update.
Useful for
Language-model RL when several candidate answers can be generated and scored.
The catch
If every answer gets the same reward, the group supplies no relative reward signal. Generating multiple answers also costs computation.
A little more detail
Why use a group?
Original GRPO uses group rewards instead of a separately trained value model, which would estimate expected future reward. That can save memory, but it does not make training free.
Is GRPO the same as RLVR?
No. RLVR supplies verifiable rewards. GRPO is one way of learning from them.
Read the research
These guides simplify the methods. The papers describe the full training recipes. Examples on this page are illustrative, not experimental results.