Humanity's Last Exam S
AI benchmark · Coding
- Built by
- Center for AI Safety
- Tier
- S why
- Snapshot
What is Humanity's Last Exam?
Humanity's Last Exam measures frontier LLMs' expert-level academic knowledge and reasoning with 2,500 closed-ended, multi-modal questions across over a hundred subjects.
Science mostly
“High accuracy on HLE would demonstrate expert-level performance on closed-ended, verifiable questions and cutting-edge scientific knowledge”
lastexam.ai/ ↗General capability
“a multi-modal benchmark at the frontier of human knowledge, designed to be the final closed-ended academic benchmark of its kind with broad subject coverage.”
lastexam.ai/ ↗
Who built Humanity's Last Exam
Led by Center for AI Safety.
Lineage
Benchmarks this one is built on and built from: versions, subsets, extensions and the indexes that combine them.
Derived benchmarks
AA-HLE extension
“AA-HLE is Artificial Analysis's run of Humanity's Last Exam, scoring models on 2,158 text-only questions without tools, graded pass@1 by a model judge.”
HLE w/o tools subset
“HLE w/o tools scores models on Humanity's Last Exam questions answered without tool access, measuring raw knowledge and reasoning.”
HLE-Diamond extension
“HLE-Diamond measures model accuracy on 1,000 refined Humanity's Last Exam questions testing expert knowledge and reasoning, scored without tools or with web and code tools.”
HLE-MultiChoice extension
“HLE-MultiChoice measures model accuracy on 513 text-only multiple-choice questions from Humanity's Last Exam, testing reasoning capabilities.”
HLE-Verified extension
“HLE-Verified is a verified and revised version of Humanity's Last Exam that evaluates frontier language models on challenging multi-domain expert questions with reduced annotation noise.”
Included in
“A composite index on the CAIS AI Dashboard that averages frontier model scores on text benchmarks such as Humanity's Last Exam (expert-level reasoning) and TextQuests (text-based video games).”
Tier S
The standard: frontier labs report it, or a trusted evaluator scores hundreds of models on it.
How much stands behind a tier: usage, who backs it, a confirmed line from its own site or paper, and a clear read on X. High with three of the four, medium with two, low otherwise.
- 478 models scored
- listed with Scale AI
- in curated RL directories (4 pts)
On X · well regarded, moderate buzz, from 9 posts about Humanity's Last Exam in the last month
The new refined HLE-Diamond version was added to Artificial Analysis's index with top models near 69%; some note score gaps are within noise.
How we got to tier S
- Adoption: 478 models scored; listed with Scale AI. Tier S.
- On X: well regarded, moderate buzz. “The new refined HLE-Diamond version was added to Artificial Analysis's index with top models near 69%; some note score gaps are within noise.” No move.
- In curated RL directories: 4 points, a floor of B.
- Confirmed: a line from its own site or paper places it.
A clear view on X moves a tier one step up or down; most entries don't move. How tiers work.
Training methods
Methods Humanity's Last Exam uses, offers or builds for, with public evidence: its own website says so, or two independent sites do.
- RLVR Reinforcement Learning with Verifiable Rewards On its own website
“Not Every Rubric Teaches Equally: Policy-Aware Rubric Rewards for RLVR”
labs.scale.com“- Open-source models fine–tuned through Reinforcement Learning with Verifiable Feedback (RLVR) exhibit significant gains over state-of-the-art, larger general-purpose models. - Text-to-SQL Benchmark: Our RL-tuned agent achieved 46.9% execution accuracy, compared to 21.9% for the best out-of-the-box baseline mode…”
labs.scale.com - GRPO Group Relative Policy Optimization On its own website
“Improving Multi-Turn Tool Use with GRPO: Results and Insights”
labs.scale.com“…nd category balance as the rubric objective while adapting criterion-level reward weights during training. POW3R uses rollout-level contrast to emphasize criteria that currently separate the policy’s outputs, making the GRPO reward more informative without changing the underlying evaluation target. Across three base policies on two datasets spanning multimodal and text-only settings, POW3R wins 24 of 30 base-policy/metric comparisons, impro…”
labs.scale.com - PPO Proximal Policy Optimization On its own website
“…lly incorrect and adjusts the importance sampling ratio for the "off-policy" trajectories in order to optimize the policy for contexts in which the hints are no longer present. We describe variants of Guide for GRPO and PPO and empirically show that Guide-GRPO on 7B and 32B parameter models improves generalization over its vanilla counterpart with up to 4% macro-average improvement across math benchmarks. We include careful ablations to an…”
labs.scale.com - SFT Supervised Fine-Tuning On its own website
“Execution accuracy by model and training method (Base / SFT / GRPO) on the T2SQL benchmark. Dashed line marks the Qwen3-Coder-480B-A35B-Instruct baseline.”
labs.scale.com
Where Humanity's Last Exam sits
Its domain on the map, live. Hover a logo for its name; click to open it. Open it on the full map
Next: browse all benchmarks. Entry from the RL Research daily scrape of public sources, 2026-11-05 snapshot.