Group Relative Policy Optimization

GRPO
Compare several attempts at the same task.

Generate several answers to one prompt. Compare their rewards within the group. Use those relative results to update the model.

The idea

You try four solutions to the same puzzle. Comparing those tries shows which did better within this practice round. It does not tell you whether the puzzle was a good test in the first place.

One prompt. Several attempts. Relative feedback.

Worked example · GRPO7 steps

Step 1 of 7

One coding task, four hidden tests.

The prompt asks for a small Python function. Four unit tests, which the model never sees, will decide how good each answer is.

Step 2 of 7

Ask for a group of four answers.

GRPO asks the current model to answer the same prompt several times. These four answers are the group, the G in GRPO. Some look right and some are clearly wrong; nobody knows yet.

Step 3 of 7

Run the tests. Reward = share of tests passed.

A program runs each answer against the four tests. A passes none, B and D pass two each, and C passes all four. So the rewards are 0, 0.5, 1 and 0.5.

Step 4 of 7

Find the group's average score.

Add the rewards and divide by four: (0 + 0.5 + 1 + 0.5) ÷ 4 = 0.5. That average is the bar every answer is measured against. PPO trains a separate value model to predict this bar; GRPO uses the group itself, which saves memory.

Step 5 of 7

Score each answer against the average.

Advantage = (reward − average) ÷ spread. C beat the group, so it gets +1.41. A fell short and gets −1.41. B and D scored exactly the average, so their advantage is 0: they are neither pushed up nor down.

Step 6 of 7

Make C-style answers more likely, A-style answers less likely.

The update raises the chance of answers with a positive advantage and lowers the chance of answers with a negative one. Then the updated model samples a fresh group for the next prompt.

Step 7 of 7

If every answer ties, there is nothing to learn.

Suppose all four answers had passed every test. Every reward would be 1, the average would be 1, and every advantage would be 0. A group of equal scores cannot say which answer was better, so GRPO learns most from prompts the model sometimes solves and sometimes misses.

Coding task code-0017

Write is_even(n): return True if n is even, otherwise False.

hidden testsis_even(2)is_even(7)is_even(0)is_even(−3)

Model same prompt × 4

Writes four different answers.

Attempt A

return n % 2 == 1

Attempt B

return n > 0

Attempt C

return n % 2 == 0

Attempt D

return n < 1

Test runner is_even(n) for n = 2, 7, 0, −3

0/4 → 0.0

2/4 → 0.5

4/4 → 1.0

2/4 → 0.5

Compare within the group - - - average = 0.5what if all pass · average = 1.0

0.0 adv -1.41

0.5 adv %!f(int=00)

1.0 adv +1.41

0.5 adv %!f(int=00)

Update · chance of writing each answer

15% ▼=

25% ==

35% ▲=

25% ==

Task or dataModel being trainedJudge or scorerGood / chosenBad / rejectedTraining update

jsontask.jsonnew at this step

{
  "id": "code-0017",
  "prompt": "Write is_even(n): return True if n is even, otherwise False.",
  "tests": [
    "is_even(2) == True",
    "is_even(7) == False",
    "is_even(0) == True",
    "is_even(-3) == False"
  ]
}

jsongroup.jsonlnew at this step

{"id": "code-0017", "attempt": "A", "completion": "return n % 2 == 1"}
{"id": "code-0017", "attempt": "B", "completion": "return n > 0"}
{"id": "code-0017", "attempt": "C", "completion": "return n % 2 == 0"}
{"id": "code-0017", "attempt": "D", "completion": "return n < 1"}

jsonresults.jsonlnew at this step

{"attempt": "A", "passed": [0, 0, 0, 0], "reward": 0.0}
{"attempt": "B", "passed": [1, 0, 0, 1], "reward": 0.5}
{"attempt": "C", "passed": [1, 1, 1, 1], "reward": 1.0}
{"attempt": "D", "passed": [0, 1, 1, 0], "reward": 0.5}

1 means the test passed. Reward = tests passed ÷ 4. In Python, -3 % 2 is 1, which is why A fails the last test too.

pythonadvantage.pynew at this step

import statistics
rewards = [0.0, 0.5, 1.0, 0.5]
mean = statistics.mean(rewards)     # 0.5
std = statistics.pstdev(rewards)    # 0.35, how spread out they are

pythonadvantage.pynew at this step

rewards = [0.0, 0.5, 1.0, 0.5]
mean, std = 0.5, 0.354
advantages = [(r - mean) / std for r in rewards]
# A: -1.41   B: 0.0   C: +1.41   D: 0.0

jsontraining batchnew at this step

{"completion": "return n % 2 == 1", "advantage": -1.41}
{"completion": "return n > 0", "advantage": 0.0}
{"completion": "return n % 2 == 0", "advantage": 1.41}
{"completion": "return n < 1", "advantage": 0.0}

Like PPO, GRPO limits how far one update can move each chance. The original version also adds a penalty for drifting far from a frozen reference copy of the model.

pythonadvantage.py · what ifnew at this step

rewards = [1.0, 1.0, 1.0, 1.0]
mean, std = 1.0, 0.0
advantages = [(r - mean) / (std + 1e-4) for r in rewards]
# [0.0, 0.0, 0.0, 0.0]: no signal from this group

Trainers add a tiny number such as 1e-4 to the spread so they never divide by zero.

Illustrative attempts and probabilities, not benchmark results. The advantage formula is the one from the original GRPO paper; production trainers add details such as clipping and a penalty for drifting from a reference model.

How GRPO works

Sample a group
Ask the current model for several answers to the same prompt.
Score the answers
Use a verifier or another chosen reward source.
Compare within the group
Estimate how each answer performed relative to its peers.
Update and repeat
Adjust the policy using those relative signals and constraints on the update.

Useful for

Language-model RL when several candidate answers can be generated and scored.

The catch

If every answer gets the same reward, the group supplies no relative reward signal. Generating multiple answers also costs computation.

A little more detail

Why use a group?

Original GRPO uses group rewards instead of a separately trained value model, which would estimate expected future reward. That can save memory, but it does not make training free.

Is GRPO the same as RLVR?

No. RLVR supplies verifiable rewards. GRPO is one way of learning from them.

Who works with GRPO

RL environment builders and AI benchmarks with public evidence that they use GRPO: their own website says so, or two independent sites do. Only confirmed evidence is shown; each page lists the quotes.

RL environments 15

AfterQuery
Evidence on its own site
“Not all tasks are equally useful for RL. Tasks the model solves consistently or fails every time provide no learning signal. We select tasks where the SFT checkpoint has a 10–80% solve rate. These are the tasks where GRPO can compute meaningful advantages: some attempts succeed, some fail, and the model can learn from the difference.”
afterquery.com
Bespoke Labs
Evidence on its own site
“We are excited to share some recent findings which show that RL can improve multi-step tool use capabilities, a first milestone. Using GRPO, the core algorithm behind DeepSeek-R1, we improved Qwen2.5-7B-Instruct’s tool use performance by 23% on a subset of the BFCL benchmark using only 100 training samples. The tasks in this benchmark require an agent to or…”
bespokelabs.ai
Daytona
Evidence on its own site
“…pt per step) 2. Evaluate : Each completion runs in its own Daytona sandbox against a test suite 3. Reward : Completions that pass more tests get higher rewards; errors or banned patterns get negative rewards 4. Update : GRPO reinforces completions that scored above their group average”
daytona.io
E2B
Evidence on its own site
“…umbing data generation through training so that we have visibility into how each task and its provenance actually influences the models we train. On the infrastructure side, running RL with a policy gradient method like GRPO or CISPO means orchestrating thousands of concurrent rollouts at once."”
e2b.dev
Halluminate
Evidence on its own site
“Training has two phases: SFT followed by RL, specifically Group Relative Policy Optimization (GRPO).”
halluminate.ai
HUD
Evidence on its own site
“The fastest way to understand GRPO is to watch a model get better at something you can see. So: take 2048, wrap it as an environment, and train a small model to play it. The whole thing is about 100 lines. The RL parts (environment contract and training…”
hud.ai
Pareto
Evidence on its own site
“To explore whether learning efficiency is a trainable property, we fine-tuned Llama 3.1 8B-IT using offline GRPO on more than 5,000 trajectories from 29 biology training tasks, with the cumulative trajectory metric as the reward target. The resulting model demonstrated improved learning efficiency on 14 of 21 held-out tasks, which…”
pareto.ai
Prime Intellect
Evidence on its own site
“A training algorithm in prime-rl is configured under [orchestrator.algo] , where type names the algorithm ( grpo , opd , sft , …) and the class defaults are its vetted setting. It has two parts:”
docs.primeintellect.ai
Proximal
Evidence on its own site
“So far, we have found that using agentic judges to check the qualitative differences between rollouts leading to positive rewards and rollouts leading to negative rewards is useful: GRPO maximizes the likelihood of positive rollouts while minimizing the likelihood of negative rollouts, so it is intuitive that tasks in which positive rewards are caused by fundamentally different reasoning strategies rath…”
proximal.ai
Refresh
Evidence on its own site
“We trained rnj-1-instruct 8B using the terminus-2 harness with our Gauntlet 4K RLVR dataset. After GRPO showed instability during training, we switched to the DAPO algorithm which provided consistent convergence.”
refresh.dev
Scale AI
Evidence on its own site
“…enuinely new knowledge). Based on this key insight, our method Guide introduces a "teacher" that provides targeted hints only when a "student" model fails all of its attempts at a problem. The training algorithm ( Guide-GRPO ) then uses off-policy corrections to ensure the student learns to solve problems independently, without guidance, at test time. - What We Found: - We confirmed that learning in RLVR is primarily driven by self-distilla…”
scale.com
Surge AI
Evidence on its own site
“…fic work that real jobs demand. Frontier models such as GPT-5.2 and Claude Opus 4.6 solve fewer than 30% of tasks when all expert-authored rubric criteria must be satisfied. Using this environment, we train GLM 4.6 with Group Relative Policy Optimization (GRPO) and adaptive clipping. After a single epoch of training, the model improves from 25.37% to 36.76% task pass rate on held-out evaluation tasks. More importantly, these gains transfer to out-of-distribution benchma…”
surgehq.ai
Ulam
Evidence on its own site
“Training views: SFT, DPO, critic and repair data, RLVR/GRPO tasks, process-reward steps, private evaluation, and regression tests.”
ulam.ai
Veris AI
Evidence on its own site
“GRPO (Group Relative Policy Optimization) uses the Veris simulation environment as a live training ground. The model generates completions, executes them against mock services and simulated users, and learns from reward sign…”
docs.veris.ai
Vmax
Evidence on its own site
“The full unix-ctf loop: harvest a validated library, instantiate multi-flag containers, train with GRPO, then evaluate on held-out and external benchmarks.”
vmax.ai

AI benchmarks 105

~1k JavaScript coding tasks in Harbor format
Evidence on its own site
“The full unix-ctf loop: harvest a validated library, instantiate multi-flag containers, train with GRPO, then evaluate on held-out and external benchmarks.”
vmax.ai
AdvancedIF - arXiv
Evidence on its own site
“…fic work that real jobs demand. Frontier models such as GPT-5.2 and Claude Opus 4.6 solve fewer than 30% of tasks when all expert-authored rubric criteria must be satisfied. Using this environment, we train GLM 4.6 with Group Relative Policy Optimization (GRPO) and adaptive clipping. After a single epoch of training, the model improves from 25.37% to 36.76% task pass rate on held-out evaluation tasks. More importantly, these gains transfer to out-of-distribution benchma…”
surgehq.ai
Agent-environment papers
Evidence on its own site
“Improving Multi-Turn Tool Use with GRPO: Results and Insights”
labs.scale.com
Antidote and Hemingway-bench leaderboards
Evidence on its own site
“…fic work that real jobs demand. Frontier models such as GPT-5.2 and Claude Opus 4.6 solve fewer than 30% of tasks when all expert-authored rubric criteria must be satisfied. Using this environment, we train GLM 4.6 with Group Relative Policy Optimization (GRPO) and adaptive clipping. After a single epoch of training, the model improves from 25.37% to 36.76% task pass rate on held-out evaluation tasks. More importantly, these gains transfer to out-of-distribution benchma…”
surgehq.ai
App-Bench
Evidence on its own site
“Not all tasks are equally useful for RL. Tasks the model solves consistently or fails every time provide no learning signal. We select tasks where the SFT checkpoint has a 10–80% solve rate. These are the tasks where GRPO can compute meaningful advantages: some attempts succeed, some fail, and the model can learn from the difference.”
afterquery.com
AppBench
Evidence on its own site
“Not all tasks are equally useful for RL. Tasks the model solves consistently or fails every time provide no learning signal. We select tasks where the SFT checkpoint has a 10–80% solve rate. These are the tasks where GRPO can compute meaningful advantages: some attempts succeed, some fail, and the model can learn from the difference.”
afterquery.com
Assemble Bench (robotics
Evidence on its own site
“The fastest way to understand GRPO is to watch a model get better at something you can see. So: take 2048, wrap it as an environment, and train a small model to play it. The whole thing is about 100 lines. The RL parts (environment contract and training…”
hud.ai
Attune
Evidence on its own site
“To explore whether learning efficiency is a trainable property, we fine-tuned Llama 3.1 8B-IT using offline GRPO on more than 5,000 trajectories from 29 biology training tasks, with the cumulative trajectory metric as the reward target. The resulting model demonstrated improved learning efficiency on 14 of 21 held-out tasks, which…”
pareto.ai
AutoResearchExam
Evidence on its own site
“We are excited to share some recent findings which show that RL can improve multi-step tool use capabilities, a first milestone. Using GRPO, the core algorithm behind DeepSeek-R1, we improved Qwen2.5-7B-Instruct’s tool use performance by 23% on a subset of the BFCL benchmark using only 100 training samples. The tasks in this benchmark require an agent to or…”
bespokelabs.ai
BankingBench
Evidence on its own site
“GRPO (Group Relative Policy Optimization) uses the Veris simulation environment as a live training ground. The model generates completions, executes them against mock services and simulated users, and learns from reward sign…”
docs.veris.ai
Bespoke Curator
Evidence on its own site
“We are excited to share some recent findings which show that RL can improve multi-step tool use capabilities, a first milestone. Using GRPO, the core algorithm behind DeepSeek-R1, we improved Qwen2.5-7B-Instruct’s tool use performance by 23% on a subset of the BFCL benchmark using only 100 training samples. The tasks in this benchmark require an agent to or…”
bespokelabs.ai
Bespoke-MiniCheck
Evidence on its own site
“We are excited to share some recent findings which show that RL can improve multi-step tool use capabilities, a first milestone. Using GRPO, the core algorithm behind DeepSeek-R1, we improved Qwen2.5-7B-Instruct’s tool use performance by 23% on a subset of the BFCL benchmark using only 100 training samples. The tasks in this benchmark require an agent to or…”
bespokelabs.ai
Bespoke-Nimble-9B model
Evidence on its own site
“We are excited to share some recent findings which show that RL can improve multi-step tool use capabilities, a first milestone. Using GRPO, the core algorithm behind DeepSeek-R1, we improved Qwen2.5-7B-Instruct’s tool use performance by 23% on a subset of the BFCL benchmark using only 100 training samples. The tasks in this benchmark require an agent to or…”
bespokelabs.ai
BlenderBench
Evidence on its own site
“We trained rnj-1-instruct 8B using the terminus-2 harness with our Gauntlet 4K RLVR dataset. After GRPO showed instability during training, we switched to the DAPO algorithm which provided consistent convergence.”
refresh.dev
BrowserBench (2025-11-22):
Evidence on its own site
“Imitate, Then Explore: SFT and GRPO”
halluminate.ai
Chartography - arXiv:2608.10677
Evidence on its own site
“…fic work that real jobs demand. Frontier models such as GPT-5.2 and Claude Opus 4.6 solve fewer than 30% of tasks when all expert-authored rubric criteria must be satisfied. Using this environment, we train GLM 4.6 with Group Relative Policy Optimization (GRPO) and adaptive clipping. After a single epoch of training, the model improves from 25.37% to 36.76% task pass rate on held-out evaluation tasks. More importantly, these gains transfer to out-of-distribution benchma…”
surgehq.ai
computer-1 computer-use harness in Harbor
Evidence on its own site
“We trained rnj-1-instruct 8B using the terminus-2 harness with our Gauntlet 4K RLVR dataset. After GRPO showed instability during training, we switched to the DAPO algorithm which provided consistent convergence.”
refresh.dev
ConjectureBench: 14,865 open math problems
Evidence on its own site
“We are excited to share some recent findings which show that RL can improve multi-step tool use capabilities, a first milestone. Using GRPO, the core algorithm behind DeepSeek-R1, we improved Qwen2.5-7B-Instruct’s tool use performance by 23% on a subset of the BFCL benchmark using only 100 training samples. The tasks in this benchmark require an agent to or…”
bespokelabs.ai
CoreCraft
Evidence on its own site
“…fic work that real jobs demand. Frontier models such as GPT-5.2 and Claude Opus 4.6 solve fewer than 30% of tasks when all expert-authored rubric criteria must be satisfied. Using this environment, we train GLM 4.6 with Group Relative Policy Optimization (GRPO) and adaptive clipping. After a single epoch of training, the model improves from 25.37% to 36.76% task pass rate on held-out evaluation tasks. More importantly, these gains transfer to out-of-distribution benchma…”
surgehq.ai
Cross-Benchmark Generalization in Long-Horizon Agents - arXiv
Evidence on its own site
“…fic work that real jobs demand. Frontier models such as GPT-5.2 and Claude Opus 4.6 solve fewer than 30% of tasks when all expert-authored rubric criteria must be satisfied. Using this environment, we train GLM 4.6 with Group Relative Policy Optimization (GRPO) and adaptive clipping. After a single epoch of training, the model improves from 25.37% to 36.76% task pass rate on held-out evaluation tasks. More importantly, these gains transfer to out-of-distribution benchma…”
surgehq.ai
Cross-Benchmark Transfer from RL on Agentic Coding Tasks - arXiv
Evidence on its own site
“…fic work that real jobs demand. Frontier models such as GPT-5.2 and Claude Opus 4.6 solve fewer than 30% of tasks when all expert-authored rubric criteria must be satisfied. Using this environment, we train GLM 4.6 with Group Relative Policy Optimization (GRPO) and adaptive clipping. After a single epoch of training, the model improves from 25.37% to 36.76% task pass rate on held-out evaluation tasks. More importantly, these gains transfer to out-of-distribution benchma…”
surgehq.ai
data-eng-bench
Evidence on its own site
“We are excited to share some recent findings which show that RL can improve multi-step tool use capabilities, a first milestone. Using GRPO, the core algorithm behind DeepSeek-R1, we improved Qwen2.5-7B-Instruct’s tool use performance by 23% on a subset of the BFCL benchmark using only 100 training samples. The tasks in this benchmark require an agent to or…”
bespokelabs.ai
DAYJOB (Healthcare/Finance) - arXiv:2610.01306
Evidence on its own site
“…fic work that real jobs demand. Frontier models such as GPT-5.2 and Claude Opus 4.6 solve fewer than 30% of tasks when all expert-authored rubric criteria must be satisfied. Using this environment, we train GLM 4.6 with Group Relative Policy Optimization (GRPO) and adaptive clipping. After a single epoch of training, the model improves from 25.37% to 36.76% task pass rate on held-out evaluation tasks. More importantly, these gains transfer to out-of-distribution benchma…”
surgehq.ai
DealTrace Bench (2026-05/06):
Evidence on its own site
“Training has two phases: SFT followed by RL, specifically Group Relative Policy Optimization (GRPO).”
halluminate.ai
Design Arena — 3D Design
Evidence on its own site
“The team then used a GRPO algorithm integrating agentic reward feedback (GRPO-AR) to perform reinforcement learning. GRPO-AR combines agent-assigned scores in three categories deemed essential for visual coding tasks – executability, static aest…”
notes.designarena.ai
Design Arena — Agentic Game Development
Evidence on its own site
“The team then used a GRPO algorithm integrating agentic reward feedback (GRPO-AR) to perform reinforcement learning. GRPO-AR combines agent-assigned scores in three categories deemed essential for visual coding tasks – executability, static aest…”
notes.designarena.ai
Design Arena — Frontend Web Apps
Evidence on its own site
“The team then used a GRPO algorithm integrating agentic reward feedback (GRPO-AR) to perform reinforcement learning. GRPO-AR combines agent-assigned scores in three categories deemed essential for visual coding tasks – executability, static aest…”
notes.designarena.ai
Design Arena — HTML Game Development
Evidence on its own site
“The team then used a GRPO algorithm integrating agentic reward feedback (GRPO-AR) to perform reinforcement learning. GRPO-AR combines agent-assigned scores in three categories deemed essential for visual coding tasks – executability, static aest…”
notes.designarena.ai
Design Arena — HTML Slides
Evidence on its own site
“The team then used a GRPO algorithm integrating agentic reward feedback (GRPO-AR) to perform reinforcement learning. GRPO-AR combines agent-assigned scores in three categories deemed essential for visual coding tasks – executability, static aest…”
notes.designarena.ai
Design Arena — Image Generation
Evidence on its own site
“The team then used a GRPO algorithm integrating agentic reward feedback (GRPO-AR) to perform reinforcement learning. GRPO-AR combines agent-assigned scores in three categories deemed essential for visual coding tasks – executability, static aest…”
notes.designarena.ai
Design Arena — Image to Web App
Evidence on its own site
“The team then used a GRPO algorithm integrating agentic reward feedback (GRPO-AR) to perform reinforcement learning. GRPO-AR combines agent-assigned scores in three categories deemed essential for visual coding tasks – executability, static aest…”
notes.designarena.ai
Design Arena — Image to Website
Evidence on its own site
“The team then used a GRPO algorithm integrating agentic reward feedback (GRPO-AR) to perform reinforcement learning. GRPO-AR combines agent-assigned scores in three categories deemed essential for visual coding tasks – executability, static aest…”
notes.designarena.ai
Design Arena — Logo Design
Evidence on its own site
“The team then used a GRPO algorithm integrating agentic reward feedback (GRPO-AR) to perform reinforcement learning. GRPO-AR combines agent-assigned scores in three categories deemed essential for visual coding tasks – executability, static aest…”
notes.designarena.ai
Design Arena — UI Components
Evidence on its own site
“The team then used a GRPO algorithm integrating agentic reward feedback (GRPO-AR) to perform reinforcement learning. GRPO-AR combines agent-assigned scores in three categories deemed essential for visual coding tasks – executability, static aest…”
notes.designarena.ai
Design Arena — Video Generation
Evidence on its own site
“The team then used a GRPO algorithm integrating agentic reward feedback (GRPO-AR) to perform reinforcement learning. GRPO-AR combines agent-assigned scores in three categories deemed essential for visual coding tasks – executability, static aest…”
notes.designarena.ai
Design Arena — Web Design Overall
Evidence on its own site
“The team then used a GRPO algorithm integrating agentic reward feedback (GRPO-AR) to perform reinforcement learning. GRPO-AR combines agent-assigned scores in three categories deemed essential for visual coding tasks – executability, static aest…”
notes.designarena.ai
Design Arena — Website Builders
Evidence on its own site
“The team then used a GRPO algorithm integrating agentic reward feedback (GRPO-AR) to perform reinforcement learning. GRPO-AR combines agent-assigned scores in three categories deemed essential for visual coding tasks – executability, static aest…”
notes.designarena.ai
DrugDiscoveryBench with Phylo
Evidence on its own site
“Improving Multi-Turn Tool Use with GRPO: Results and Insights”
labs.scale.com
Dynamic Semantic Tags technical report with Intuit
Evidence on its own site
“We are excited to share some recent findings which show that RL can improve multi-step tool use capabilities, a first milestone. Using GRPO, the core algorithm behind DeepSeek-R1, we improved Qwen2.5-7B-Instruct’s tool use performance by 23% on a subset of the BFCL benchmark using only 100 training samples. The tasks in this benchmark require an agent to or…”
bespokelabs.ai
EnterpriseBench CoreCraft - arXiv:2602.16179
Evidence on its own site
“- CoreCraft (enterprise RL Environment): GRPO with rubric-based rewards moved GLM 4.6 by +11.4pp on held-out tasks and transferred out-of-distribution: +4.5pp on BFCL Parallel, +7.4pp on τ²-Bench Retail, +6.8pp on Toolathlon — while Pass³, a reliability metric requ…”
surgehq.ai
Evalchemy
Evidence on its own site
“We are excited to share some recent findings which show that RL can improve multi-step tool use capabilities, a first milestone. Using GRPO, the core algorithm behind DeepSeek-R1, we improved Qwen2.5-7B-Instruct’s tool use performance by 23% on a subset of the BFCL benchmark using only 100 training samples. The tasks in this benchmark require an agent to or…”
bespokelabs.ai
FinanceQA
Evidence on its own site
“Not all tasks are equally useful for RL. Tasks the model solves consistently or fails every time provide no learning signal. We select tasks where the SFT checkpoint has a 10–80% solve rate. These are the tasks where GRPO can compute meaningful advantages: some attempts succeed, some fail, and the model can learn from the difference.”
afterquery.com
FrontierSWE
Evidence on its own site
“So far, we have found that using agentic judges to check the qualitative differences between rollouts leading to positive rewards and rollouts leading to negative rewards is useful: GRPO maximizes the likelihood of positive rollouts while minimizing the likelihood of negative rollouts, so it is intuitive that tasks in which positive rewards are caused by fundamentally different reasoning strategies rath…”
proximal.ai
FrontierSWE (v1)
Evidence on its own site
“So far, we have found that using agentic judges to check the qualitative differences between rollouts leading to positive rewards and rollouts leading to negative rewards is useful: GRPO maximizes the likelihood of positive rollouts while minimizing the likelihood of negative rollouts, so it is intuitive that tasks in which positive rewards are caused by fundamentally different reasoning strategies rath…”
proximal.ai
Gauntlet 4K RLVR dataset, access by request
Evidence on its own site
“We trained rnj-1-instruct 8B using the terminus-2 harness with our Gauntlet 4K RLVR dataset. After GRPO showed instability during training, we switched to the DAPO algorithm which provided consistent convergence.”
refresh.dev
GDP.pdf - arXiv:2607.11192
Evidence on its own site
“- CoreCraft (enterprise RL Environment): GRPO with rubric-based rewards moved GLM 4.6 by +11.4pp on held-out tasks and transferred out-of-distribution: +4.5pp on BFCL Parallel, +7.4pp on τ²-Bench Retail, +6.8pp on Toolathlon — while Pass³, a reliability metric requ…”
surgehq.ai
GDP.xlsx
Evidence on its own site
“- CoreCraft (enterprise RL Environment): GRPO with rubric-based rewards moved GLM 4.6 by +11.4pp on held-out tasks and transferred out-of-distribution: +4.5pp on BFCL Parallel, +7.4pp on τ²-Bench Retail, +6.8pp on Toolathlon — while Pass³, a reliability metric requ…”
surgehq.ai
General Agent synthetic environment
Evidence on its own site
“A training algorithm in prime-rl is configured under [orchestrator.train.algo] , where type names the algorithm ( grpo , opd , sft , …) and the class defaults are its vetted setting. It has two parts:”
docs.primeintellect.ai
GEPA
Evidence on its own site
“We are excited to share some recent findings which show that RL can improve multi-step tool use capabilities, a first milestone. Using GRPO, the core algorithm behind DeepSeek-R1, we improved Qwen2.5-7B-Instruct’s tool use performance by 23% on a subset of the BFCL benchmark using only 100 training samples. The tasks in this benchmark require an agent to or…”
bespokelabs.ai
HANDBOOK.md - arXiv:2607.25398
Evidence on its own site
“- CoreCraft (enterprise RL Environment): GRPO with rubric-based rewards moved GLM 4.6 by +11.4pp on held-out tasks and transferred out-of-distribution: +4.5pp on BFCL Parallel, +7.4pp on τ²-Bench Retail, +6.8pp on Toolathlon — while Pass³, a reliability metric requ…”
surgehq.ai
HANDBOOK.md Agents
Evidence on its own site
“- CoreCraft (enterprise RL Environment): GRPO with rubric-based rewards moved GLM 4.6 by +11.4pp on held-out tasks and transferred out-of-distribution: +4.5pp on BFCL Parallel, +7.4pp on τ²-Bench Retail, +6.8pp on Toolathlon — while Pass³, a reliability metric requ…”
surgehq.ai
HiL-Bench
Evidence on its own site
“Improving Multi-Turn Tool Use with GRPO: Results and Insights”
labs.scale.com
Human Evaluation
Evidence on its own site
“- CoreCraft (enterprise RL Environment): GRPO with rubric-based rewards moved GLM 4.6 by +11.4pp on held-out tasks and transferred out-of-distribution: +4.5pp on BFCL Parallel, +7.4pp on τ²-Bench Retail, +6.8pp on Toolathlon — while Pass³, a reliability metric requ…”
surgehq.ai
Humanity's Last Exam
Evidence on its own site
“Improving Multi-Turn Tool Use with GRPO: Results and Insights”
labs.scale.com
IDE-Bench
Evidence on its own site
“Not all tasks are equally useful for RL. Tasks the model solves consistently or fails every time provide no learning signal. We select tasks where the SFT checkpoint has a 10–80% solve rate. These are the tasks where GRPO can compute meaningful advantages: some attempts succeed, some fail, and the model can learn from the difference.”
afterquery.com
INTELLECT-2
Evidence on its own site
“A training algorithm in prime-rl is configured under [orchestrator.train.algo] , where type names the algorithm ( grpo , opd , sft , …) and the class defaults are its vetted setting. It has two parts:”
docs.primeintellect.ai
INTELLECT-3 Technical Report
Evidence on its own site
“A training algorithm in prime-rl is configured under [orchestrator.train.algo] , where type names the algorithm ( grpo , opd , sft , …) and the class defaults are its vetted setting. It has two parts:”
docs.primeintellect.ai
Kubernetes Is the Wrong Primitive for Diverse Workloads
Evidence on its own site
“…pt per step) 2. Evaluate : Each completion runs in its own Daytona sandbox against a test suite 3. Reward : Completions that pass more tests get higher rewards; errors or banned patterns get negative rewards 4. Update : GRPO reinforces completions that scored above their group average”
daytona.io
Leap
Evidence on its own site
“To explore whether learning efficiency is a trainable property, we fine-tuned Llama 3.1 8B-IT using offline GRPO on more than 5,000 trajectories from 29 biology training tasks, with the cumulative trajectory metric as the reward target. The resulting model demonstrated improved learning efficiency on 14 of 21 held-out tasks, which…”
pareto.ai
Legora BAR
Evidence on its own site
“Not all tasks are equally useful for RL. Tasks the model solves consistently or fails every time provide no learning signal. We select tasks where the SFT checkpoint has a 10–80% solve rate. These are the tasks where GRPO can compute meaningful advantages: some attempts succeed, some fail, and the model can learn from the difference.”
afterquery.com
LiveWeb samples (with Browserbase) - 2026-09-22/28
Evidence on its own site
“The fastest way to understand GRPO is to watch a model get better at something you can see. So: take 2048, wrap it as an environment, and train a small model to play it. The whole thing is about 100 lines. The RL parts (environment contract and training…”
hud.ai
Market-Bench
Evidence on its own site
“Not all tasks are equally useful for RL. Tasks the model solves consistently or fails every time provide no learning signal. We select tasks where the SFT checkpoint has a 10–80% solve rate. These are the tasks where GRPO can compute meaningful advantages: some attempts succeed, some fail, and the model can learn from the difference.”
afterquery.com
Math RL Tasks
Evidence on its own site
“Training views: SFT, DPO, critic and repair data, RLVR/GRPO tasks, process-reward steps, private evaluation, and regression tests.”
ulam.ai
MCP Atlas
Evidence on its own site
“Improving Multi-Turn Tool Use with GRPO: Results and Insights”
labs.scale.com
Measuring Autonomous AI Research
Evidence on its own site
“A training algorithm in prime-rl is configured under [orchestrator.train.algo] , where type names the algorithm ( grpo , opd , sft , …) and the class defaults are its vetted setting. It has two parts:”
docs.primeintellect.ai
OpenReward
Evidence on its own site
“- Set up distributed RL training with SkyRL - Configure an OpenReward environment for training - Monitor training progress with WandB and OpenReward rollouts - Train a model on the WhoDunit environment using GRPO”
docs.openreward.ai
OpenThoughts
Evidence on its own site
“We are excited to share some recent findings which show that RL can improve multi-step tool use capabilities, a first milestone. Using GRPO, the core algorithm behind DeepSeek-R1, we improved Qwen2.5-7B-Instruct’s tool use performance by 23% on a subset of the BFCL benchmark using only 100 training samples. The tasks in this benchmark require an agent to or…”
bespokelabs.ai
OpenThoughts datasets
Evidence on its own site
“We are excited to share some recent findings which show that RL can improve multi-step tool use capabilities, a first milestone. Using GRPO, the core algorithm behind DeepSeek-R1, we improved Qwen2.5-7B-Instruct’s tool use performance by 23% on a subset of the BFCL benchmark using only 100 training samples. The tasks in this benchmark require an agent to or…”
bespokelabs.ai
OSWorld Windows tasks on Daytona Windows sandboxes
Evidence on its own site
“…pt per step) 2. Evaluate : Each completion runs in its own Daytona sandbox against a test suite 3. Reward : Completions that pass more tests get higher rewards; errors or banned patterns get negative rewards 4. Update : GRPO reinforces completions that scored above their group average”
daytona.io
OSWorld-Verified
Evidence on its own site
“The fastest way to understand GRPO is to watch a model get better at something you can see. So: take 2048, wrap it as an environment, and train a small model to play it. The whole thing is about 100 lines. The RL parts (environment contract and training…”
hud.ai
Post-Training on Office Work Improves Software Engineering - arXiv
Evidence on its own site
“- CoreCraft (enterprise RL Environment): GRPO with rubric-based rewards moved GLM 4.6 by +11.4pp on held-out tasks and transferred out-of-distribution: +4.5pp on BFCL Parallel, +7.4pp on τ²-Bench Retail, +6.8pp on Toolathlon — while Pass³, a reliability metric requ…”
surgehq.ai
PRBench
Evidence on its own site
“Improving Multi-Turn Tool Use with GRPO: Results and Insights”
labs.scale.com
Production-debugging RL environment (hud-ops-diagnostics) - 2026-01-20
Evidence on its own site
“The fastest way to understand GRPO is to watch a model get better at something you can see. So: take 2048, wrap it as an environment, and train a small model to play it. The whole thing is about 100 lines. The RL parts (environment contract and training…”
hud.ai
PROPEL
Evidence on its own site
“The full unix-ctf loop: harvest a validated library, instantiate multi-flag containers, train with GRPO, then evaluate on held-out and external benchmarks.”
vmax.ai
READY or Not enterprise agent deployment benchmarks
Evidence on its own site
“Improving Multi-Turn Tool Use with GRPO: Results and Insights”
labs.scale.com
Remote Labor Index
Evidence on its own site
“Improving Multi-Turn Tool Use with GRPO: Results and Insights”
labs.scale.com
RFT case study with Savi Security, Qwen3 4B scam detection
Evidence on its own site
“Supervised Fine-Tuning. Examples. Overview. Configuration. veris.yaml ... GRPO (Group Relative Policy Optimization) uses the Veris simulation ...”
docs.veris.ai
Riemann-Bench - arXiv:2604.06802
Evidence on its own site
“- CoreCraft (enterprise RL Environment): GRPO with rubric-based rewards moved GLM 4.6 by +11.4pp on held-out tasks and transferred out-of-distribution: +4.5pp on BFCL Parallel, +7.4pp on τ²-Bench Retail, +6.8pp on Toolathlon — while Pass³, a reliability metric requ…”
surgehq.ai
Robustly Evaluating Cyber Capabilities
Evidence on its own site
“So far, we have found that using agentic judges to check the qualitative differences between rollouts leading to positive rewards and rollouts leading to negative rewards is useful: GRPO maximizes the likelihood of positive rollouts while minimizing the likelihood of negative rollouts, so it is intuitive that tasks in which positive rewards are caused by fundamentally different reasoning strategies rath…”
proximal.ai
RSI Bench
Evidence on its own site
“Improving Multi-Turn Tool Use with GRPO: Results and Insights”
labs.scale.com
Rubric-based RL papers
Evidence on its own site
“Improving Multi-Turn Tool Use with GRPO: Results and Insights”
labs.scale.com
Scale Labs (launched 2026-03-09
Evidence on its own site
“Improving Multi-Turn Tool Use with GRPO: Results and Insights”
labs.scale.com
Scaling Agentic RL: 365k+ SWE/terminal/search tasks
Evidence on its own site
“A training algorithm in prime-rl is configured under [orchestrator.train.algo] , where type names the algorithm ( grpo , opd , sft , …) and the class defaults are its vetted setting. It has two parts:”
docs.primeintellect.ai
Searching Over Sandbox States for Coding Agents
Evidence on its own site
“…pt per step) 2. Evaluate : Each completion runs in its own Daytona sandbox against a test suite 3. Reward : Completions that pass more tests get higher rewards; errors or banned patterns get negative rewards 4. Update : GRPO reinforces completions that scored above their group average”
daytona.io
SheetBench-50 (released 2025-10-01)
Evidence on its own site
“The fastest way to understand GRPO is to watch a model get better at something you can see. So: take 2048, wrap it as an environment, and train a small model to play it. The whole thing is about 100 lines. The RL parts (environment contract and training…”
hud.ai
SheetBench-50 (with Sepal AI) - 2025-10-01
Evidence on its own site
“The fastest way to understand GRPO is to watch a model get better at something you can see. So: take 2048, wrap it as an environment, and train a small model to play it. The whole thing is about 100 lines. The RL parts (environment contract and training…”
hud.ai
Simulated-to-Real Transfer for Web Agents (2026-07-27):
Evidence on its own site
“Imitate, Then Explore: SFT and GRPO”
halluminate.ai
SpreadsheetBench 2
Evidence on its own site
“We improved Qwen3.8-27B’s DeepSWE pass@1 (avg 3) from 27.8% to 39.1% (+11.3 points) using GRPO on 500 tasks from AfterQuery’s SWE agent training dataset. AfterQuery builds all tasks from real software engineering workflows within private, enterprise-grade codebases to avoid evaluation contamination and ensure rea…”
afterquery.com
SWE-bench Pro
Evidence on its own site
“Improving Multi-Turn Tool Use with GRPO: Results and Insights”
labs.scale.com
SWE-Bench Pro - Private Dataset
Evidence on its own site
“Improving Multi-Turn Tool Use with GRPO: Results and Insights”
labs.scale.com
SWE-Bench Pro v1 (2025-09-19) and v2
Evidence on its own site
“Improving Multi-Turn Tool Use with GRPO: Results and Insights”
labs.scale.com
Systematic Reward Hacking study
Evidence on its own site
“A training algorithm in prime-rl is configured under [orchestrator.train.algo] , where type names the algorithm ( grpo , opd , sft , …) and the class defaults are its vetted setting. It has two parts:”
docs.primeintellect.ai
Terminal-Bench 2.0
Evidence on its own site
“We are excited to share some recent findings which show that RL can improve multi-step tool use capabilities, a first milestone. Using GRPO, the core algorithm behind DeepSeek-R1, we improved Qwen2.5-7B-Instruct’s tool use performance by 23% on a subset of the BFCL benchmark using only 100 training samples. The tasks in this benchmark require an agent to or…”
bespokelabs.ai
The Hierarchy of Agentic Capabilities - arXiv:2601.09032
Evidence on its own site
“- CoreCraft (enterprise RL Environment): GRPO with rubric-based rewards moved GLM 4.6 by +11.4pp on held-out tasks and transferred out-of-distribution: +4.5pp on BFCL Parallel, +7.4pp on τ²-Bench Retail, +6.8pp on Toolathlon — while Pass³, a reliability metric requ…”
surgehq.ai
TRAIL
Evidence on its own site
“…sm (TP4·EP4·ETP2) on 1 node of 8×B200; for RL, rollouts are served by SGLang on a second node of 8×B200 (disaggregated, so serving never competes with the trainer for memory); orchestrated by miles/slime . RL algorithm: GRPO, 16 samples per prompt. Reward: how far Mario travels through level 1-1, minus a time penalty, plus a flag bonus. One caveat up front: "NVFP4" is selective , not applied everywhere. Only the routed MoE expert weights ar…”
patronus.ai
True Agents Model the World / ECHO
Evidence on its own site
“A training algorithm in prime-rl is configured under [orchestrator.train.algo] , where type names the algorithm ( grpo , opd , sft , …) and the class defaults are its vetted setting. It has two parts:”
docs.primeintellect.ai
Tuesday Work Index leaderboard
Evidence on its own site
“- CoreCraft (enterprise RL Environment): GRPO with rubric-based rewards moved GLM 4.6 by +11.4pp on held-out tasks and transferred out-of-distribution: +4.5pp on BFCL Parallel, +7.4pp on τ²-Bench Retail, +6.8pp on Toolathlon — while Pass³, a reliability metric requ…”
surgehq.ai
UI-Bench
Evidence on its own site
“We improved Qwen3.8-27B’s DeepSWE pass@1 (avg 3) from 27.8% to 39.1% (+11.3 points) using GRPO on 500 tasks from AfterQuery’s SWE agent training dataset. AfterQuery builds all tasks from real software engineering workflows within private, enterprise-grade codebases to avoid evaluation contamination and ensure rea…”
afterquery.com
Universal offline sandbox escape disclosure
Evidence on its own site
“A training algorithm in prime-rl is configured under [orchestrator.train.algo] , where type names the algorithm ( grpo , opd , sft , …) and the class defaults are its vetted setting. It has two parts:”
docs.primeintellect.ai
unix-ctf
Evidence on its own site
“The full unix-ctf loop: harvest a validated library, instantiate multi-flag containers, train with GRPO, then evaluate on held-out and external benchmarks.”
vmax.ai
VADER
Evidence on its own site
“We improved Qwen3.8-27B’s DeepSWE pass@1 (avg 3) from 27.8% to 39.1% (+11.3 points) using GRPO on 500 tasks from AfterQuery’s SWE agent training dataset. AfterQuery builds all tasks from real software engineering workflows within private, enterprise-grade codebases to avoid evaluation contamination and ensure rea…”
afterquery.com
Web Bench (2025-05-30; ~5
Evidence on its own site
“Imitate, Then Explore: SFT and GRPO”
halluminate.ai
WebBench
Evidence on its own site
“Imitate, Then Explore: SFT and GRPO”
halluminate.ai
Westworld Finance Diligence Bench (2026-08-11):
Evidence on its own site
“Imitate, Then Explore: SFT and GRPO”
halluminate.ai
Westworld v1 simulated-web environments (2025-11-23):
Evidence on its own site
“Training has two phases: SFT followed by RL, specifically Group Relative Policy Optimization (GRPO).”
halluminate.ai