JobBench B
AI benchmark · Agentic
What is JobBench?
Current benchmarks for occupational AI agents are scoped primarily by economic values, telling a replacement story. We introduce JobBench, which evaluates AI agents on the workflows that experts identify as high-priority for delegation, empowering humans based on their needs instead of replacing them with GDP value. JobBench covers 130 agentic tasks across 35 occupations. Each task is packaged as a workspace of heterogeneous reference files, requiring the agent to reason through the cluttered information streams of real professional work. Outputs are graded by a fact-anchored chain of rubrics, averaging 35.6 binary criteria per task. We evaluate 36 models; the strongest, Claude Opus~4.7 under Claude Code, reaches only 45.9 %. We hope JobBench shifts the community's target labour-market effect from replacement to enhancement: building agents that do what humans actually want delegated, not only what is most economically valuable.
Tier B
Established: 15 or more models scored, or curated RL directories rate it.
How much stands behind a tier: usage, who backs it, a confirmed line from its own site or paper, and a clear read on X. High with three of the four, medium with two, low otherwise.
- 48 models scored
- in curated RL directories (2 pts)
On X · unclear, low buzz, from 2 posts about JobBench in the last month
JobBench appears only in passing, apparently as an out-of-distribution test in Google's RRSI harness-evolution paper, with no direct discussion of its quality.
How we got to tier B
- Adoption: 48 models scored. Tier B.
- On X: unclear, low buzz. “JobBench appears only in passing, apparently as an out-of-distribution test in Google's RRSI harness-evolution paper, with no direct discussion of its quality.” No move.
- In curated RL directories: 2 points, a floor of C.
- Not yet confirmed by a line from its own site or paper.
A clear view on X moves a tier one step up or down; most entries don't move. How tiers work.
Not on the map yet: no line from its own site or paper places it in a domain.
Next: browse all benchmarks. Entry from the RL Research daily scrape of public sources, 2026-10-11 snapshot.