Chartography - arXiv:2608.10677 C

Name: Chartography - arXiv:2608.10677
Creator: Surge AI
Keywords: Enterprise Workflows,Long-Horizon

AI benchmark · Enterprise Workflows

Publisher: Surge AI
Domains: Enterprise Workflows Long-Horizon
On X: @HelloSurgeAI ↗
Tier: C why
Snapshot: October 11, 2026

Open benchmark ↗surgehq.ai/research

Latest from Surge AI on X

@HelloSurgeAI ↗

An AI agent found every expiring rent discount in a commercial real estate workbook. Then it missed three break clauses representing roughly $14 million in forecast rent. The task required following two different paths through a 15-tab workbook. The model followed one and stopped. Knowing when you’re done is part of the job.
— HelloSurgeAI (@HelloSurgeAI) Oct 9, 2026
Introducing sudo L7: Can Coding Agents Act Like Staff Engineers? Coding agents are getting very good at solving well-defined problems. But staff engineers do much more than write correct code. They understand unfamiliar systems, make architectural decisions, spot risks the ticket never mentioned, and know when passing the tests isn't enough. sudo L7 measures that judgment. Our new software engineering benchmark contains 60 expert-authored tasks, primarily in private production codebases from real companies, complete with legacy code, technical debt, and messy dependencies. We evaluate eight dimensions of engineering behavior, from functional correctness and engineering craft to practical judgment, thought partnership, and knowing when to push back. The best-performing agents score under ~45%. Coding is only part of the job. See you next cycle. Explore sudo L7:
— HelloSurgeAI (@HelloSurgeAI) Oct 8, 2026
Only 2 of 66 runs caught a $25.7M pricing error. One screenshot. One unit mistake. A 100× overstatement. See what else frontier models miss on DAYJOB: Finance:
— HelloSurgeAI (@HelloSurgeAI) Oct 6, 2026

What is Chartography - arXiv:2608.10677?

Chartography measures whether frontier multimodal models can correctly read and reason over professional, domain-specific charts from medicine, engineering, finance, manufacturing, and the sciences.

Media
“Failures concentrate in visual perception: models can miss nuanced features, misread values along sparsely labeled axes, mishandle projected 3D geometry”
arxiv.org/abs/2608.10677 ↗
Health and bio
“We test whether frontier models can read the charts professionals make decisions from in medicine, engineering, finance, manufacturing, and the sciences.”
surgehq.ai/research ↗
Finance
“We test whether frontier models can read the charts professionals make decisions from in medicine, engineering, finance, manufacturing, and the sciences.”
surgehq.ai/research ↗
Science
“Professionals across medicine, engineering, finance, manufacturing, and the sciences often make consequential decisions from charts.”
arxiv.org/abs/2608.10677 ↗

Paper: arxiv.org/abs/2608.10677 ↗

Tier C

Niche: a handful of models scored, or built by an RL vendor.

How much stands behind a tier: usage, who backs it, a confirmed line from its own site or paper, and a clear read on X. High with three of the four, medium with two, low otherwise.

built by Surge AI
in curated RL directories (2 pts)

How we got to tier C

Adoption: built by Surge AI. Tier C.
Not yet read on X.
In curated RL directories: 2 points, a floor of C.
Confirmed: a line from its own site or paper places it.

A clear view on X moves a tier one step up or down; most entries don't move. How tiers work.

Where Chartography - arXiv:2608.10677 sits

Its realm in the universe, live. Hover a star for its name; click to open it. Open it in the universe

The galaxy could not load. See every vendor on the board.

More Enterprise Workflows AI benchmarks

Enterprise Workflows RL environments

Next: browse all benchmarks. Entry from the RL Research daily scrape of public sources, 2026-10-11 snapshot.