Math benchmarks for LLMs and RL environments for math

Covers math“Math & Reasoning” source ↗

Sells RL environments“Turn any task into an RL environment. Init, develop, eval, and push with the Prime CLI.” source ↗

Covers math“doctors, lawyers, investment bankers, Fields Medalists” source ↗

Sells RL environments“We create rich, complex RL environments that challenge agentic models in novel ways, and design verifiers that reward their behavior.” source ↗

Covers math“we deployed a similar approach for math, fostering a dedicated community of IMO medalists” source ↗

Sells RL environments“To further scale these ambitions, we sell the environments we make to frontier labs.” source ↗

Covers math“Our work is grounded in open mathematical research, where progress is measured by the quality of reasoning, not just final answers.” source ↗

Sells RL environments“Train and evaluate agents in stateful, verifier-backed research environments where progress, revision, and failure become learning signals.” source ↗

Tests math“All 30 problems from the 2025 American Invitational Mathematics Examination, testing olympiad-level mathematical reasoning with integer answers from 000-999.” source ↗

Tests math“covers a wide range of mathematical topics, including algebra, geometry, and number theory.” source ↗

Tests math“GSM8K (Grade school math word problems)” source ↗

Tests math“we introduce MATH, a new dataset of 12,500 challenging competition mathematics problems.” source ↗

Tests math“The International Mathematical Olympiad (IMO) is a six-problem, two-day proof-based competition for pre-university students.” source ↗

Tests math“8.9 RiemannBench” source ↗

Tests math“MATH 500 Algebra, probability, trigonometry problems” source ↗

Tests math“Apex-Shortlist (no tools) | 74.9 | 28.9 | 71.1 | 77.4 | 61.4 | 85.8 | 82.4” source ↗

Tests math“8.7 USAMO 2026” source ↗

Tests math“ProofBench evaluates whether models can translate mathematical reasoning into machine-checkable proofs.” source ↗

Tests math“present MathArena Apex: a set of 12 hard problems drawn from most public final-answer competitions held in 2025.” source ↗

Tests math“ArXivMath is a final-answer benchmark of research-level mathematics maintained by MathArena.” source ↗

Tests math“The Harvard-MIT Mathematics Tournament (HMMT) is one of the largest and most prestigious high school math competitions in the United States.” source ↗

Tests math“IMO 2025 & USAMO 2026: Aligned with MathArena official evaluation. Each contest: 6 problems, max score 42.” source ↗

Tests math“we introduce MATH, a new dataset of 12,500 challenging competition mathematics problems.” source ↗

Tests math“we introduce GSM8K, a dataset of 8.5K high quality linguistically diverse grade school math word problems.” source ↗

Tests math“This dataset contains the questions from AIME 2026 used for the MathArena Leaderboard” source ↗

Tests math“Research-level mathematics problems at the frontier of current AI capabilities” source ↗

Tests math“Tier 4 (hardest) FrontierMath problems - research-level difficulty” source ↗

Tests math“Competition-level math problems from OTIS Mock AIME evaluating olympiad-level math” source ↗

Tests math“Math RL Tasks is a public collection of Ulam's mathematical construction problems for reinforcement learning with verifiable rewards (RLVR) and model evaluation.” source ↗

Tests math“IMO-AnswerBench first tests models on 400 diverse Olympiad problems with verifiable short answers.” source ↗

Tests math“the first benchmark for evaluating reward models in formal theorem proving with Lean 4” source ↗

Tests math“the **IneqMath** benchmark of Olympiad-level inequality problems.” source ↗

Tests math“a comprehensive and challenging benchmark specifically designed to assess LLMs' mathematical reasoning at the Olympiad level” source ↗

Tests math“PutnamBench is a benchmark for evaluation of theorem-proving algorithms on competition mathematics problems sourced from the William Lowell Putnam Mathematical Competition years 1962 - 2025.” source ↗

Tests math“We also introduce GeoSym-Bench, an expert-curated suite of 511 complex samples for rigorous evaluation.” source ↗

Tests math“we introduce GSM8K, a dataset of 8.5K high quality linguistically diverse grade school math word problems.” source ↗

Tests math“This dataset contains the questions from HMMT February 2026 used for the MathArena Leaderboard” source ↗

Tests math“IMO-AnswerBench first tests models on 400 diverse Olympiad problems with verifiable short answers.” source ↗

Tests math“800 high-quality questions covering 350 theorems (e.g. Taylor's theorem, Lagrange's theorem, Huffman coding, Quantum Theorem, Elasticity Theorem, etc) from Math, Physics, EE&CS, and Finance.” source ↗

Tests math“This benchmark tests a model’s ability to solve challenging mathematics problems by generating step-by-step solutions and providing the correct final answer.” source ↗

Tests math“A benchmark of several hundred unpublished, highly challenging mathematics problems.” source ↗

Tests math“Exceptionally difficult research-level math problems.” source ↗

No line from its own site or paper places it in Math yet; the catalog lists it here.

No line from its own site or paper places it in Math yet; the catalog lists it here.

Tests math“RO reformulation remains challenging because it requires precise multi-step reasoning and mathematically consistent transformations.” source ↗

Tests math“We introduce CAM-Bench, a Lean 4 theorem-proving benchmark of 1,000 Lean proof targets in computational and applied mathematics” source ↗

Tests math“CNMO 2024 (Pass@1):” source ↗

Tests math“FormInv: A Measurement Protocol for Semantic Invariance in Mathematical Reasoning Benchmarks” source ↗

Tests math“among the first benchmarks to systematically evaluate LLM-based efficient algorithm design for realistic large-scale optimization problems” source ↗

Tests math“Math · Interactive Proof\ \ HLE Math Problems\ \ 52 instances · Best: Grok 76.9%” source ↗

Tests math“We curate 515 challenging pre-engineering mathematics, physics and chemistry problems from the highly competitive IIT JEE-Advanced exam.” source ↗

Tests math“a diagnostic benchmark evaluating 10 frontier large language models on structured linear algebra computation across a strict dimensional gradient of 3x3, 4x4, and 5x5 matrices” source ↗

Tests math“a dynamic multiple-choice benchmark for research-level mathematical reasoning built from recent arXiv papers published after model training cutoffs” source ↗

Tests math“we introduce M3Kang, the first massively multilingual, multimodal mathematical reasoning dataset for VLMs.” source ↗

Tests math“the first large-scale autoformalization benchmark of in the wild graduate-level mathematics” source ↗

Tests math“we identify such components and introduce ProofRank, a benchmark curated from challenging mathematical competitions.” source ↗

Tests math“featuring 8,476 problems from Olympiad-level mathematics and physics competitions, including the Chinese college entrance exam.” source ↗

No line from its own site or paper places it in Math yet; the catalog lists it here.

Tests math“ProofNet is a benchmark for autoformalization and formal proving of undergraduate-level mathematics.” source ↗

Tests math“we introduce Soohak, a 439-problem benchmark newly authored from scratch by 64 mathematicians.” source ↗

Tests math“To enable more robust evaluation of automatic MWP solvers, we created a challenge set called "SVAMP".” source ↗

Tests math“Type: percentageMath” source ↗

Tests math“This dataset contains the questions from AIME 2023 I used for the MathArena Leaderboard” source ↗

Tests math“**What a high score indicates:** difficult high-school competition-math insight and precise multi-step calculation.” source ↗

Tests math“It also locked in Tier 4, a private set of 43 research-level problems built to break even the best models.” source ↗

Tests math“On Beyond AIME (69.1), which requires deeper reasoning chains and harder mathematical decomposition, the model leads or matches the comparison set.” source ↗

Tests math“Figure 13: Example of zero-shot CoT generated by MetaMath-7B when prompted with an example from MGSM Bengali.” source ↗

Tests math“This dataset contains the questions from BRUMO 2025 used for the MathArena Leaderboard” source ↗

Tests math“American Invitational Mathematics Examination (AIME) 2026” source ↗

Tests math“MathArena’s June2026 ArXivMath evaluation, using49research-mathematics problems and averaging four runs per problem without tools.” source ↗

Tests math“On research math, Anthropic's system card has Opus 5.5 at 91.2% on ArXivMath at max” source ↗

Tests math“We present the Chinese Elementary School Math Word Problems (CMATH) dataset, comprising 1.7k elementary school-level math word problems with detailed annotations” source ↗

Tests math“American Invitational Mathematics Examination (AIME) 2025” source ↗

Tests math“HMMT Feb 2023 | MathArena/hmmt\_feb\_2023 | 30 | Math expression” source ↗

Tests math“Harvard-MIT Mathematics Tournament February 2024.” source ↗

Tests math“High-school mathematics competition; AIRank’s HMMT25 protocol is not publicly disclosed” source ↗

No line from its own site or paper places it in Math yet; the catalog lists it here.

Tests math“**Task Type**: Competition Mathematics Problem Solving” source ↗

Tests math“Task:MathArena Hmmt Feb 2026” source ↗

Tests math“Also in **Math**” source ↗

Tests math“Type: numericalMath” source ↗

Tests math“By renaming identifiers in the Natural Number Game in Lean 4, we created a zero-knowledge, closed environment.” source ↗

Tests math“An internal holdout set of competition math problems.” source ↗

Tests math“we construct a new benchmark consisting of 98,734 theorems and proofs extracted from Lean's math library.” source ↗

Tests math“MiniF2F is a formal mathematics benchmark (translated across multiple formal systems) consisting of exercise statements from olympiads (AMC, AIME, IMO)” source ↗

Tests math“Our main testbed is equation-suffix prediction: the predictor sees context and the first part of a displayed equation, then forecasts the rest.” source ↗

Tests math“We present two new components, FATE-H and FATE-X, each with 100 problems in abstract and commutative algebra.” source ↗

Tests math“This benchmark contains 100 extremely challenging abstract algebra and commutative algebra exercises collected from:” source ↗

Tests math“We have rewritten the relevant fragment of the MATH benchmark into its functional variant MATH(), with functionalization of other benchmarks to follow.” source ↗

Tests math“MAWPS : A Math Word Problem Repository” source ↗

Tests math“We introduce the Multilingual Grade School Math (MGSM) benchmark, by manually translating 250 grade-school math problems from the GSM8K dataset” source ↗

Tests math“we further introduce NL4COP, a benchmark of 150 instances across 50 diverse problem types with more complex decision logic, more tightly coupled constraints” source ↗

Tests math“we introduce PolyMath, a multilingual mathematical reasoning benchmark covering 18 languages and 4 easy-to-hard difficulty levels.” source ↗

Tests math“derives five tracks of evaluation benchmarks, measuring different aspects of auto-formalization and theorem-proving capabilities.” source ↗

Sells data“Start with reviewed reasoning datasets or build targeted data packs around your model's actual errors, tasks, and evaluation results.” source ↗

Covers math“Our work is grounded in open mathematical research, where progress is measured by the quality of reasoning, not just final answers.” source ↗

Sells data“Aggregating all human research data” source ↗

Covers math“we deployed a similar approach for math, fostering a dedicated community of IMO medalists” source ↗

Sells data“We produce exclusive and off-the-shelf (OTS) datasets built around multi-step hierarchical reasoning, tool-augmented workflows, and domain-specific problems” source ↗

Covers math“SciCode, FrontierScience, FrontierMath, CritPt Benchmark Training Sets” source ↗

Sells data“We look for the most brilliant minds on the planet in every domain – doctors, lawyers, investment bankers, Fields Medalists, Harvard professors” source ↗

Covers math“doctors, lawyers, investment bankers, Fields Medalists” source ↗

Sells data“Private evals, and data for SWE, Tau, MLE, MMMU, and more” source ↗

Covers math“Physics, chemistry, math, and biology” source ↗

Sells data“Request dataset samples or join our global expert network to advance superintelligence.” source ↗

Covers math“For physics, chemistry, math, biology, and more.” source ↗