← All methodologies

Reinforcement Learning from Human Feedback

RLHF
People show what “better” means.

People compare answers. A scoring model learns those preferences. The assistant then practices getting higher scores.

The idea

A teacher compares two essays and picks the clearer one. After seeing many choices, a teaching assistant learns how to grade. The student can now practice with that assistant's feedback.

First learn the score. Then learn from it.

Worked example · RLHF6 steps

Step 1 of 6

The model writes two answers to one prompt.

We start from a model that can already chat, usually after SFT. For each prompt, it writes two different answers.

Step 2 of 6

A person picks the better answer.

A human labeler reads both and picks A, because it fits a younger student. Choosing between two answers is easier and more consistent than giving one answer a score out of 10.

Step 3 of 6

Repeat for thousands of prompts.

Many labelers compare many pairs. The InstructGPT reward model, an early ChatGPT-style system, learned from comparisons on about 33,000 prompts.

Step 4 of 6

Train a reward model to predict the choices.

A reward model reads a prompt and an answer and outputs one number. Training pushes the chosen answer’s number above the rejected one’s. Before training it scores A at 0.1 and B at 0.2, the wrong order. After training: A 1.8, B −0.6.

Step 5 of 6

The assistant practices and the reward model grades.

Now the reward model can grade answers nobody labeled. The assistant answers a new prompt, gets a score of 1.2, and an RL algorithm, usually PPO, makes high-scoring answers more likely. A penalty keeps the assistant from drifting too far from where it started.

Step 6 of 6

People check the result.

A higher reward-model score is not proof of a better answer. An assistant can learn to please the reward model with answers that sound good but are wrong. So people compare the old and new assistant on fresh prompts.

Prompt

Explain rain to a younger student.

Answer A

Rain happens when tiny water drops in clouds join up and get too heavy to float.

chosen
Answer B

Atmospheric precipitation results from hydrometeor growth.

rejected
Human labeler

“A is better for a younger student.”

Comparisons

about 33,000 prompts with human choices

Reward model
A before 1.8
B before -0.6
RL practice · new prompt: “Why is the sky blue?”
Air scatters blue light much more than red, so blue reaches your eyes from all over the sky. reward model: 1.2 PPO update ▲
People check

Compare assistant v1 and v2 on fresh prompts. A high reward-model score can still hide a wrong answer.

Task or dataModel being trainedJudge or scorerGood / chosenBad / rejectedFrozen or hidden

jsonto_label.jsonl · one rownew at this step

{
  "prompt": "Explain rain to a younger student.",
  "answer_a": "Rain happens when tiny water drops in clouds join up and get too heavy to float.",
  "answer_b": "Atmospheric precipitation results from hydrometeor growth."
}

jsoncomparisons.jsonl · one rownew at this step

{
  "prompt": "Explain rain to a younger student.",
  "chosen": "Rain happens when tiny water drops in clouds join up and get too heavy to float.",
  "rejected": "Atmospheric precipitation results from hydrometeor growth.",
  "labeler": "human"
}

jsoncomparisons.jsonlnew at this step

{"prompt": "Explain rain to a younger student.", "chosen": "Rain happens when…", "rejected": "Atmospheric precipitation…"}
{"prompt": "Write a haiku about Monday.", "chosen": "…", "rejected": "…"}
{"prompt": "Is raw cookie dough safe to eat?", "chosen": "…", "rejected": "…"}
…

pythonreward_model_loss.pynew at this step

loss = -math.log(sigmoid(r_chosen - r_rejected))
# before: r_A = 0.1, r_B = 0.2   → loss 0.74
# after:  r_A = 1.8, r_B = -0.6  → loss 0.09

jsonrollouts.jsonl · one rownew at this step

{
  "prompt": "Why is the sky blue?",
  "response": "Sunlight has every color. Air scatters blue light much more than red, so blue reaches your eyes from all over the sky.",
  "reward": 1.2
}

jsonhuman_eval.jsonl · one rownew at this step

{
  "prompt": "Explain tides to a 10-year-old.",
  "answer_1": "assistant-v1",
  "answer_2": "assistant-v2",
  "human_prefers": "answer_2"
}
Illustrative answers and scores. This is the classic reward-model RLHF setup described in the InstructGPT paper; labelers there ranked several answers per prompt rather than only two. People label training examples; they do not grade every later practice answer.

How RLHF works

  1. Collect comparisons

    Ask people which of two answers they prefer.

  2. Train a scoring model

    A reward model learns to predict those choices.

  3. Practice and update

    The assistant generates answers. An RL algorithm uses the scores to adjust it.

  4. Check with people again

    Test whether higher scores actually produced more helpful answers.

Useful for

Helpful explanations, tone, and other goals without a single answer key.

The catch

A higher predicted preference score is not proof of truth. People and reward models can both favor a convincing but wrong answer.

A little more detail

Two words worth knowing

Reward means a training score, not a feeling. A reward model is another model trained to give that score.

One important distinction

RLHF describes the role of human feedback. PPO is one algorithm that can perform the updates. They are not competing names for the same thing.

Who works with RLHF

RL environment builders and AI benchmarks with public evidence that they use RLHF: their own website says so, or two independent sites do. Only confirmed evidence is shown; each page lists the quotes.

RL environments 13

  • Contra Labs
    Evidence on its own site
    “Creative Human Data or Contra's creative RLHF (reinforcement learning from human feedback) uses professional creatives—like designers, writers, and marketers—to evaluate and fine-tune AI model outputs based on subjective quality, style, and brand fit. Contra is set…”
    contralabs.com
  • Gray Swan AI
    Evidence on its own site
    “From automated testing, human experts, and crowdsourced intelligence, you need a continuous stream of adversarial findings to inform guardrail design, RLHF, and post-training safety work.”
    grayswan.ai
  • micro1
    Evidence on its own site
    “Most of this work falls under Reinforcement Learning from Human Feedback (RLHF) — where experts review AI outputs, rank responses, and flag errors. You're not building AI. You're setting the standard it learns from.”
    micro1.ai
  • Parsewave
    Evidence on its own site
    “The most effective approach is to use curated, real-world engineering tasks that represent a reliable ground truth. Parsewave datasets are designed for use in supervised fine-tuning, reinforcement learning from human feedback, or as benchmarks to assess model performance.”
    parsewave.ai
  • Revelo
    Evidence on its own site
    “Experts compare, rank, and rate your model’s outputs for RLHF, DPO, and reward models.”
    revelo.com
  • Rise Data Labs
    Evidence on its own site
    “Unmatched Expert Data for Superior SFT and RLHF”
    risedatalabs.com
  • Scale AI
    Evidence on its own site
    “Text2SQL systems promise to democratize access to enterprise data but often fail to handle the complexity of real-world database queries, even if they perform well on test datasets. We found that Reinforcement Learning from Human Feedback (RLHF) is a viable approach for active learning from incorrect production queries to improve Text2SQL accuracy.”
    scale.com
  • Snorkel AI
    Evidence on its own site
    “Our work explored two techniques for model alignment: RLHF and direct preference optimization (DPO).”
    snorkel.ai
  • Surge AI
    Evidence on its own site
    “Overview: Anthropic is One of the World’s Leading AI and LLM Companies The Problem: Building Trustworthy, High-Quality Human Feedback The Solution: Rich Human Feedback via Surge AI’s RLHF Platform Results: Anthropic’s Claude – a Safe, Highly Capable, State-of-the-Art AI Assistant”
    surgehq.ai
  • Toloka
    Evidence on its own site
    “ContentsCollect RLHF preference data”
    toloka.ai
  • Turing
    Evidence on its own site
    “Business Analyst (RLHF/Audio) - Arabic”
    work.turing.com
  • Uber AI
    Evidence on its own site
    “Tap into our global network of contributors and specialized experts to generate supervised fine-tuning (SFT) data and execute reinforcement learning and human feedback (RLHF) workflows.”
    uber.com
  • Wirestock
    Evidence on its own site
    “Some evaluator jobs are listed as RLHF roles. RLHF stands for reinforcement learning with human feedback. In plain terms, reviewers help AI systems learn which output a person would choose, and why.”
    wirestock.io

AI benchmarks 67

  • AdvancedIF - arXiv
    Evidence on its own site
    “Overview: Anthropic is One of the World’s Leading AI and LLM Companies The Problem: Building Trustworthy, High-Quality Human Feedback The Solution: Rich Human Feedback via Surge AI’s RLHF Platform Results: Anthropic’s Claude – a Safe, Highly Capable, State-of-the-Art AI Assistant”
    surgehq.ai
  • Agent Red Teaming (ART) benchmark
    Evidence on its own site
    “From automated testing, human experts, and crowdsourced intelligence, you need a continuous stream of adversarial findings to inform guardrail design, RLHF, and post-training safety work.”
    grayswan.ai
  • Agents' Last Exam
    Evidence on its own site
    “Our work explored two techniques for model alignment: RLHF and direct preference optimization (DPO).”
    snorkel.ai
  • AI Models Now Introduce Safety Risks in the Physical World
    Evidence on its own site
    “Most of this work falls under Reinforcement Learning from Human Feedback (RLHF) — where experts review AI outputs, rank responses, and flag errors. You're not building AI. You're setting the standard it learns from.”
    micro1.ai
  • Antidote and Hemingway-bench leaderboards
    Evidence on its own site
    “Overview: Anthropic is One of the World’s Leading AI and LLM Companies The Problem: Building Trustworthy, High-Quality Human Feedback The Solution: Rich Human Feedback via Surge AI’s RLHF Platform Results: Anthropic’s Claude – a Safe, Highly Capable, State-of-the-Art AI Assistant”
    surgehq.ai
  • Arena AI Document
    Evidence on its own site
    “1. We curate high quality ground truth preference pairs from Chatbot Arena battles as well as existing verifiable correctness benchmarks. 2. We experimentally correlate metrics on each benchmark to downstream RLHF-ed LLMs. 3. We fully open-source PPE, the resulting comprehensive benchmark for reward models with metrics directly linked to downstream RLHF outcomes.”
    arena.ai
  • Arena AI Image Edit
    Evidence on its own site
    “1. We curate high quality ground truth preference pairs from Chatbot Arena battles as well as existing verifiable correctness benchmarks. 2. We experimentally correlate metrics on each benchmark to downstream RLHF-ed LLMs. 3. We fully open-source PPE, the resulting comprehensive benchmark for reward models with metrics directly linked to downstream RLHF outcomes.”
    arena.ai
  • Arena AI Image-to-Video
    Evidence on its own site
    “1. We curate high quality ground truth preference pairs from Chatbot Arena battles as well as existing verifiable correctness benchmarks. 2. We experimentally correlate metrics on each benchmark to downstream RLHF-ed LLMs. 3. We fully open-source PPE, the resulting comprehensive benchmark for reward models with metrics directly linked to downstream RLHF outcomes.”
    arena.ai
  • Arena AI Text-to-Image
    Evidence on its own site
    “1. We curate high quality ground truth preference pairs from Chatbot Arena battles as well as existing verifiable correctness benchmarks. 2. We experimentally correlate metrics on each benchmark to downstream RLHF-ed LLMs. 3. We fully open-source PPE, the resulting comprehensive benchmark for reward models with metrics directly linked to downstream RLHF outcomes.”
    arena.ai
  • Arena AI Text-to-Video
    Evidence on its own site
    “1. We curate high quality ground truth preference pairs from Chatbot Arena battles as well as existing verifiable correctness benchmarks. 2. We experimentally correlate metrics on each benchmark to downstream RLHF-ed LLMs. 3. We fully open-source PPE, the resulting comprehensive benchmark for reward models with metrics directly linked to downstream RLHF outcomes.”
    arena.ai
  • Arena AI Video Edit
    Evidence on its own site
    “1. We curate high quality ground truth preference pairs from Chatbot Arena battles as well as existing verifiable correctness benchmarks. 2. We experimentally correlate metrics on each benchmark to downstream RLHF-ed LLMs. 3. We fully open-source PPE, the resulting comprehensive benchmark for reward models with metrics directly linked to downstream RLHF outcomes.”
    arena.ai
  • Arena AI WebDev Arena
    Evidence on its own site
    “1. We curate high quality ground truth preference pairs from Chatbot Arena battles as well as existing verifiable correctness benchmarks. 2. We experimentally correlate metrics on each benchmark to downstream RLHF-ed LLMs. 3. We fully open-source PPE, the resulting comprehensive benchmark for reward models with metrics directly linked to downstream RLHF outcomes.”
    arena.ai
  • Benchmarking Agents in Insurance Underwriting Environments
    Evidence on its own site
    “Our work explored two techniques for model alignment: RLHF and direct preference optimization (DPO).”
    snorkel.ai
  • BigLaw Bench: Research
    Evidence on its own site
    “Our work explored two techniques for model alignment: RLHF and direct preference optimization (DPO).”
    snorkel.ai
  • Chartography - arXiv:2608.10677
    Evidence on its own site
    “Overview: Anthropic is One of the World’s Leading AI and LLM Companies The Problem: Building Trustworthy, High-Quality Human Feedback The Solution: Rich Human Feedback via Surge AI’s RLHF Platform Results: Anthropic’s Claude – a Safe, Highly Capable, State-of-the-Art AI Assistant”
    surgehq.ai
  • Comparing AI Agents to Cybersecurity Professionals in Real-World Penetration Testing
    Evidence on its own site
    “From automated testing, human experts, and crowdsourced intelligence, you need a continuous stream of adversarial findings to inform guardrail design, RLHF, and post-training safety work.”
    grayswan.ai
  • Continual Learning Bench
    Evidence on its own site
    “Our work explored two techniques for model alignment: RLHF and direct preference optimization (DPO).”
    snorkel.ai
  • CoreCraft
    Evidence on its own site
    “Overview: Anthropic is One of the World’s Leading AI and LLM Companies The Problem: Building Trustworthy, High-Quality Human Feedback The Solution: Rich Human Feedback via Surge AI’s RLHF Platform Results: Anthropic’s Claude – a Safe, Highly Capable, State-of-the-Art AI Assistant”
    surgehq.ai
  • CortexRetrievalBench
    Evidence on its own site
    “Most of this work falls under Reinforcement Learning from Human Feedback (RLHF) — where experts review AI outputs, rank responses, and flag errors. You're not building AI. You're setting the standard it learns from.”
    micro1.ai
  • Crosby-micro1 RedlineBench
    Evidence on its own site
    “Most of this work falls under Reinforcement Learning from Human Feedback (RLHF) — where experts review AI outputs, rank responses, and flag errors. You're not building AI. You're setting the standard it learns from.”
    micro1.ai
  • Cross-Benchmark Generalization in Long-Horizon Agents - arXiv
    Evidence on its own site
    “Overview: Anthropic is One of the World’s Leading AI and LLM Companies The Problem: Building Trustworthy, High-Quality Human Feedback The Solution: Rich Human Feedback via Surge AI’s RLHF Platform Results: Anthropic’s Claude – a Safe, Highly Capable, State-of-the-Art AI Assistant”
    surgehq.ai
  • Cross-Benchmark Transfer from RL on Agentic Coding Tasks - arXiv
    Evidence on its own site
    “Overview: Anthropic is One of the World’s Leading AI and LLM Companies The Problem: Building Trustworthy, High-Quality Human Feedback The Solution: Rich Human Feedback via Surge AI’s RLHF Platform Results: Anthropic’s Claude – a Safe, Highly Capable, State-of-the-Art AI Assistant”
    surgehq.ai
  • DAYJOB (Healthcare/Finance) - arXiv:2610.01306
    Evidence on its own site
    “Overview: Anthropic is One of the World’s Leading AI and LLM Companies The Problem: Building Trustworthy, High-Quality Human Feedback The Solution: Rich Human Feedback via Surge AI’s RLHF Platform Results: Anthropic’s Claude – a Safe, Highly Capable, State-of-the-Art AI Assistant”
    surgehq.ai
  • EnterpriseBench CoreCraft - arXiv:2602.16179
    Evidence on its own site
    “Overview: Anthropic is One of the World’s Leading AI and LLM Companies The Problem: Building Trustworthy, High-Quality Human Feedback The Solution: Rich Human Feedback via Surge AI’s RLHF Platform Results: Anthropic’s Claude – a Safe, Highly Capable, State-of-the-Art AI Assistant”
    surgehq.ai
  • Evaluations cited in 23 frontier model system cards and reports listed on the vendor homepage (Claude family through Opus/Sonnet 5.5
    Evidence on its own site
    “From automated testing, human experts, and crowdsourced intelligence, you need a continuous stream of adversarial findings to inform guardrail design, RLHF, and post-training safety work.”
    grayswan.ai
  • FinQA / SnorkelFinance environment and benchmark
    Evidence on its own site
    “Our work explored two techniques for model alignment: RLHF and direct preference optimization (DPO).”
    snorkel.ai
  • GDP.pdf - arXiv:2607.11192
    Evidence on its own site
    “Overview: Anthropic is One of the World’s Leading AI and LLM Companies The Problem: Building Trustworthy, High-Quality Human Feedback The Solution: Rich Human Feedback via Surge AI’s RLHF Platform Results: Anthropic’s Claude – a Safe, Highly Capable, State-of-the-Art AI Assistant”
    surgehq.ai
  • GDP.xlsx
    Evidence on its own site
    “Overview: Anthropic is One of the World’s Leading AI and LLM Companies The Problem: Building Trustworthy, High-Quality Human Feedback The Solution: Rich Human Feedback via Surge AI’s RLHF Platform Results: Anthropic’s Claude – a Safe, Highly Capable, State-of-the-Art AI Assistant”
    surgehq.ai
  • GPU Glossary
    Evidence on its own site
    “Open-source reward models have become essential components of modern RLHF (Reinforcement Learning from Human Feedback) pipelines. This guide examines seven leading models and how to deploy them effectively with Modal's infrastructure.”
    modal.com
  • HANDBOOK.md - arXiv:2607.25398
    Evidence on its own site
    “Overview: Anthropic is One of the World’s Leading AI and LLM Companies The Problem: Building Trustworthy, High-Quality Human Feedback The Solution: Rich Human Feedback via Surge AI’s RLHF Platform Results: Anthropic’s Claude – a Safe, Highly Capable, State-of-the-Art AI Assistant”
    surgehq.ai
  • HANDBOOK.md Agents
    Evidence on its own site
    “Overview: Anthropic is One of the World’s Leading AI and LLM Companies The Problem: Building Trustworthy, High-Quality Human Feedback The Solution: Rich Human Feedback via Surge AI’s RLHF Platform Results: Anthropic’s Claude – a Safe, Highly Capable, State-of-the-Art AI Assistant”
    surgehq.ai
  • Human Creativity Benchmark
    Evidence on its own site
    “Creative Human Data or Contra's creative RLHF (reinforcement learning from human feedback) uses professional creatives—like designers, writers, and marketers—to evaluate and fine-tune AI model outputs based on subjective quality, style, and brand fit. Contra is set…”
    contralabs.com
  • Human Evaluation
    Evidence on its own site
    “Overview: Anthropic is One of the World’s Leading AI and LLM Companies The Problem: Building Trustworthy, High-Quality Human Feedback The Solution: Rich Human Feedback via Surge AI’s RLHF Platform Results: Anthropic’s Claude – a Safe, Highly Capable, State-of-the-Art AI Assistant”
    surgehq.ai
  • IPI Arena
    Evidence on its own site
    “From automated testing, human experts, and crowdsourced intelligence, you need a continuous stream of adversarial findings to inform guardrail design, RLHF, and post-training safety work.”
    grayswan.ai
  • Learning from Less: RLVR in Low Data Regimes
    Evidence on its own site
    “Our work explored two techniques for model alignment: RLHF and direct preference optimization (DPO).”
    snorkel.ai
  • LibraryDesignBench
    Evidence on its own site
    “Our work explored two techniques for model alignment: RLHF and direct preference optimization (DPO).”
    snorkel.ai
  • LMArena Search Arena
    Evidence on its own site
    “1. We curate high quality ground truth preference pairs from Chatbot Arena battles as well as existing verifiable correctness benchmarks. 2. We experimentally correlate metrics on each benchmark to downstream RLHF-ed LLMs. 3. We fully open-source PPE, the resulting comprehensive benchmark for reward models with metrics directly linked to downstream RLHF outcomes.”
    arena.ai
  • LMArena Text Arena
    Evidence on its own site
    “1. We curate high quality ground truth preference pairs from Chatbot Arena battles as well as existing verifiable correctness benchmarks. 2. We experimentally correlate metrics on each benchmark to downstream RLHF-ed LLMs. 3. We fully open-source PPE, the resulting comprehensive benchmark for reward models with metrics directly linked to downstream RLHF outcomes.”
    arena.ai
  • LMArena Vision Arena
    Evidence on its own site
    “1. We curate high quality ground truth preference pairs from Chatbot Arena battles as well as existing verifiable correctness benchmarks. 2. We experimentally correlate metrics on each benchmark to downstream RLHF-ed LLMs. 3. We fully open-source PPE, the resulting comprehensive benchmark for reward models with metrics directly linked to downstream RLHF outcomes.”
    arena.ai
  • LMArena WebDev Arena
    Evidence on its own site
    “1. We curate high quality ground truth preference pairs from Chatbot Arena battles as well as existing verifiable correctness benchmarks. 2. We experimentally correlate metrics on each benchmark to downstream RLHF-ed LLMs. 3. We fully open-source PPE, the resulting comprehensive benchmark for reward models with metrics directly linked to downstream RLHF outcomes.”
    arena.ai
  • LongExtractionBench
    Evidence on its own site
    “After rapidly hiring these 32 professionals we immediately hit the ground running using the proprietary micro1 human data platform. Because they were already vetted by our AI recruiter, which includes an RLHF exercise assessment, all of our newly onboarded team members were already familiar with the platform and started working within 72 hours of receiving the project requirements.”
    micro1.ai
  • NIST CAISI blog on the UK AISI x Gray Swan agent red-teaming competition
    Evidence on its own site
    “From automated testing, human experts, and crowdsourced intelligence, you need a continuous stream of adversarial findings to inform guardrail design, RLHF, and post-training safety work.”
    grayswan.ai
  • No Last Mile: A Theory of the Human Data Market
    Evidence on its own site
    “Most of this work falls under Reinforcement Learning from Human Feedback (RLHF) — where experts review AI outputs, rank responses, and flag errors. You're not building AI. You're setting the standard it learns from.”
    micro1.ai
  • OpenReward
    Evidence on its own site
    “environment = or client.environments.get(name="NVIDIA/Nemotron-RLHF-GenRM-v1") tasks = environment.list tasks(split="train") tools = environment.list tools(format="openai")”
    openreward.ai
  • OSWorld 2.0
    Evidence on its own site
    “Our work explored two techniques for model alignment: RLHF and direct preference optimization (DPO).”
    snorkel.ai
  • PII Transformation for Enterprise Datasets
    Evidence on its own site
    “Most of this work falls under Reinforcement Learning from Human Feedback (RLHF) — where experts review AI outputs, rank responses, and flag errors. You're not building AI. You're setting the standard it learns from.”
    micro1.ai
  • Post-Training on Office Work Improves Software Engineering - arXiv
    Evidence on its own site
    “Overview: Anthropic is One of the World’s Leading AI and LLM Companies The Problem: Building Trustworthy, High-Quality Human Feedback The Solution: Rich Human Feedback via Surge AI’s RLHF Platform Results: Anthropic’s Claude – a Safe, Highly Capable, State-of-the-Art AI Assistant”
    surgehq.ai
  • Quail open-source AI-SQL inference engine with CMU Full Stack Data Lab
    Evidence on its own site
    “Open-source reward models have become essential components of modern RLHF (Reinforcement Learning from Human Feedback) pipelines. This guide examines seven leading models and how to deploy them effectively with Modal's infrastructure.”
    modal.com
  • Realm
    Evidence on its own site
    “Most of this work falls under Reinforcement Learning from Human Feedback (RLHF) — where experts review AI outputs, rank responses, and flag errors. You're not building AI. You're setting the standard it learns from.”
    micro1.ai
  • Realm: Financial reasoning benchmark
    Evidence on its own site
    “Most of this work falls under Reinforcement Learning from Human Feedback (RLHF) — where experts review AI outputs, rank responses, and flag errors. You're not building AI. You're setting the standard it learns from.”
    micro1.ai
  • Realm: Legal reasoning benchmark
    Evidence on its own site
    “Most of this work falls under Reinforcement Learning from Human Feedback (RLHF) — where experts review AI outputs, rank responses, and flag errors. You're not building AI. You're setting the standard it learns from.”
    micro1.ai
  • Realm: Pathology-report reasoning benchmark
    Evidence on its own site
    “After rapidly hiring these 32 professionals we immediately hit the ground running using the proprietary micro1 human data platform. Because they were already vetted by our AI recruiter, which includes an RLHF exercise assessment, all of our newly onboarded team members were already familiar with the platform and started working within 72 hours of receiving the project requirements.”
    micro1.ai
  • Realm: Tax reasoning benchmark
    Evidence on its own site
    “Most of this work falls under Reinforcement Learning from Human Feedback (RLHF) — where experts review AI outputs, rank responses, and flag errors. You're not building AI. You're setting the standard it learns from.”
    micro1.ai
  • Riemann-Bench - arXiv:2604.06802
    Evidence on its own site
    “Overview: Anthropic is One of the World’s Leading AI and LLM Companies The Problem: Building Trustworthy, High-Quality Human Feedback The Solution: Rich Human Feedback via Surge AI’s RLHF Platform Results: Anthropic’s Claude – a Safe, Highly Capable, State-of-the-Art AI Assistant”
    surgehq.ai
  • RIFT / Hitchhiker's Guide to Rubric Quality
    Evidence on its own site
    “Our work explored two techniques for model alignment: RLHF and direct preference optimization (DPO).”
    snorkel.ai
  • Scale Labs (launched 2026-03-09
    Evidence on its own site
    “…roved representations via offline data or self-supervised objectives have shown impressive results in traditional reinforcement learning (RL). Nevertheless, it is unclear how improved representation learning can benefit reinforcement learning from human feedback (RLHF) on language models (LMs). In this work, we propose training reward models (RMs) in a contrastive, goal-conditioned fashion by increasing the representation similarity of future states along sampled preferred traj…”
    labs.scale.com
  • Senior SWE-bench
    Evidence on its own site
    “Our work explored two techniques for model alignment: RLHF and direct preference optimization (DPO).”
    snorkel.ai
  • Shrinking the Generation-Verification Gap with Weak Verifiers
    Evidence on its own site
    “Our work explored two techniques for model alignment: RLHF and direct preference optimization (DPO).”
    snorkel.ai
  • SlopCodeBench
    Evidence on its own site
    “Our work explored two techniques for model alignment: RLHF and direct preference optimization (DPO).”
    snorkel.ai
  • Snorkel: Rapid Training Data Creation with Weak Supervision
    Evidence on its own site
    “Our work explored two techniques for model alignment: RLHF and direct preference optimization (DPO).”
    snorkel.ai
  • SWE-bench Pro
    Evidence on its own site
    “Exploring reinforcement learning from human feedback (RLHF) to align language models with human preferences for safer adoption and use.”
    labs.scale.com
  • Team research listed on the research page
    Evidence on its own site
    “From automated testing, human experts, and crowdsourced intelligence, you need a continuous stream of adversarial findings to inform guardrail design, RLHF, and post-training safety work.”
    grayswan.ai
  • Terminal-Bench
    Evidence on its own site
    “Our work explored two techniques for model alignment: RLHF and direct preference optimization (DPO).”
    snorkel.ai
  • The Benchmark Ceiling
    Evidence on its own site
    “After rapidly hiring these 32 professionals we immediately hit the ground running using the proprietary micro1 human data platform. Because they were already vetted by our AI recruiter, which includes an RLHF exercise assessment, all of our newly onboarded team members were already familiar with the platform and started working within 72 hours of receiving the project requirements.”
    micro1.ai
  • The Hierarchy of Agentic Capabilities - arXiv:2601.09032
    Evidence on its own site
    “Overview: Anthropic is One of the World’s Leading AI and LLM Companies The Problem: Building Trustworthy, High-Quality Human Feedback The Solution: Rich Human Feedback via Surge AI’s RLHF Platform Results: Anthropic’s Claude – a Safe, Highly Capable, State-of-the-Art AI Assistant”
    surgehq.ai
  • Toloka Arena
    Evidence on its own site
    “ContentsCollect RLHF preference data”
    toloka.ai
  • Tuesday Work Index leaderboard
    Evidence on its own site
    “Overview: Anthropic is One of the World’s Leading AI and LLM Companies The Problem: Building Trustworthy, High-Quality Human Feedback The Solution: Rich Human Feedback via Surge AI’s RLHF Platform Results: Anthropic’s Claude – a Safe, Highly Capable, State-of-the-Art AI Assistant”
    surgehq.ai

Read the research

These guides simplify the methods. The papers describe the full training recipes. Examples on this page are illustrative, not experimental results.