Directory label
Reasoning benchmarks for LLMs and RL environments
A label other catalogs put on entries, not one of the domains on our map. On the map, each entry here sits in the domain its own site or paper leads with.
Hard reasoning problems built to resist memorization, such as graduate-level science questions and abstract visual puzzles.
- AI benchmarks
- 156
- Shares entries with
- 5 categories
Snapshot 2026-11-05. How categories are built · Where the data comes from
In short
Only benchmarks use the Reasoning tag: the RL environment directories file their vendors under other categories. Browse all RL environments.
AI benchmarks 156
-
GPQA Diamond
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
ARC-AGI-1
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
ARC-AGI-2
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
CritPt
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
AA-LCR
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
MuSR
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
LisanBench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
GeneBench-Pro
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
WildBench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Context Arena
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
FutureEval
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
ARC-AGI-3
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Terminal-Bench-Science
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Forge
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
TRACK
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
VeRA
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
EnigmaEval
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
BBH
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
HellaSwag
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
WinoGrande
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
LongBench v2
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Chartographer
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Deep20Bench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
EgoCoT-Bench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
SimpleBench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
EgoMemReason
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
EstGraph
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
LOGICAL-COMMONSENSEQA
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
LogicScore
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
MixRea
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
PlanningBench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
SDGBiasBench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
SeePhys Pro
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
StemBind
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
STEMVerse
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
TableVista
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
TriViewBench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
When Seekers Are Hard to Help
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
DROP
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
PARSE
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
ROSE
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
CorpusQA 1M
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
MRCRv2
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Graphwalks Parents 128K
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
MRCR 1M
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
A2RBench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Beyond Continuity
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
ChLogic
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
CogScale
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
CommonWhy
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
DiscoverPhysics
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Enhancing Pathological VLMs with Cross-scale Reasoning
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
LLMEval-Logic
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
PRISM
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Absurd World
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
BilliardPhys-Bench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
ChaosBench-Logic v2
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Code-Guided Reasoning for Small Language Models
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
CombEval
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Connecting the Dots
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
CounterCount
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
CulMind
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
DeepWeb-Bench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
DeFAb
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Don't Judge a Book by its Cover
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
DRInQ
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
EnactToM
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
From 0-Order Selection to 2-Order Judgment
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
GENSTRAT
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
GRACE
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
GraphARC
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
GRUFF: LLM Pronoun Fidelity, Reasoning, and Biases in German
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
GSM-SEM
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
GTEval
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
HOLMES: Evaluating Higher-Order Logical Reasoning in LLMs
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
HRBench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Kinship Data Benchmark for Multi-hop Reasoning
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
LEAF: A Living Benchmark for Event-Augmented Forecasting
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
MathConstraint
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
MINDGAMES
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
MOV-Bench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
NARRA-Gym for Evaluating Interactive Narrative Agents
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
NL-PDDL-Bench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
NLCO
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
NoisyBench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
NoisyCausal
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
NSMQ Riddles
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
OmniToM
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
ORAgentBench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
PCD Benchmark
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
PIVOTS
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
QMFOLBench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
ReplaySCM
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
RMCB
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
SciPaths: Forecasting Pathways to Scientific Discovery
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
StressEval
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
TeleResilienceBench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Text2Opt-Bench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
The Complexity Ceiling Benchmark
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
The Metanym Game
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
TraceEval
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
TriggerBench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
TS-Skill
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
TSCognition
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
WMW (World Models in Words)
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
XCR-Bench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
XDomainBench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
ARC-AGI-3 Provider Adapter
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
STEMBench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Human Behavior Bench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Game Arena
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
GIM
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
FACTS Grounding V1
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
LongMemEval
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
GraphWalks BFS 256K–1M
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Conceptual Reasoning
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Pencil Puzzle Bench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
MRCR v2 64K-128K
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
MRCR v2 128K-256K
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
MRCR v2 256K-512K
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
MRCR v2 512K-1M
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
AI-Needle
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
MLCR-AA
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
BullshitBench v2
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
PropensityBench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
NoLiMa
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
NoLiMa-Hard
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Long Context Reasoning (AA-LCR) Extended
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Long Context Reasoning (AA-LCR v1.1)
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
FACTS Grounding v2
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
BeliefTrack
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Can Language Model Agents be Helpful Circuit Explainers in Mechanistic Interpretability?
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Beyond Accuracy
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Block-R1
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
ChaosBench-Logic
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Diagnosing and Mitigating Sycophancy and Skepticism in LLM Causal Judgment
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
DiagnosticIQ
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
ForeSci
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
GeoBuildBench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
GR-Ben
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
How Does Personalized Memory Shape LLM Behavior? Benchmarking Rational Preference Utilization in Personalized Assistants
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
MIMeBench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
OracleProto
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Physics Question Scene Graph
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
ProjectionBench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
Robust Asynchronous Planning via Auto-Formalization
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
SCRuBEval
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
The Abstraction Gap in Vision-Language Causal Reasoning
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
VAIR
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
-
XLGoBench
No line from its own site or paper places it in Reasoning yet; the catalog lists it here.
6 not yet confirmedListed in the catalog; no line from its own site or paper confirms it yet.
Next: pick another domain. Tags come from the RL Research daily scrape, canonicalized nightly.