HANDBOOK.md - arXiv:2607.25398 C

Name: HANDBOOK.md - arXiv:2607.25398
Creator: Surge AI
Keywords: Enterprise Workflows,Long-Horizon

AI benchmark · Enterprise Workflows

Publisher: Surge AI
Domains: Enterprise Workflows Long-Horizon
On X: @HelloSurgeAI ↗
Tier: C why
Snapshot: October 11, 2026

Open benchmark ↗surgehq.ai/research

Latest from Surge AI on X

@HelloSurgeAI ↗

An AI agent found every expiring rent discount in a commercial real estate workbook. Then it missed three break clauses representing roughly $14 million in forecast rent. The task required following two different paths through a 15-tab workbook. The model followed one and stopped. Knowing when you’re done is part of the job.
— HelloSurgeAI (@HelloSurgeAI) Oct 9, 2026
Introducing sudo L7: Can Coding Agents Act Like Staff Engineers? Coding agents are getting very good at solving well-defined problems. But staff engineers do much more than write correct code. They understand unfamiliar systems, make architectural decisions, spot risks the ticket never mentioned, and know when passing the tests isn't enough. sudo L7 measures that judgment. Our new software engineering benchmark contains 60 expert-authored tasks, primarily in private production codebases from real companies, complete with legacy code, technical debt, and messy dependencies. We evaluate eight dimensions of engineering behavior, from functional correctness and engineering craft to practical judgment, thought partnership, and knowing when to push back. The best-performing agents score under ~45%. Coding is only part of the job. See you next cycle. Explore sudo L7:
— HelloSurgeAI (@HelloSurgeAI) Oct 8, 2026
Only 2 of 66 runs caught a $25.7M pricing error. One screenshot. One unit mistake. A 100× overstatement. See what else frontier models miss on DAYJOB: Finance:
— HelloSurgeAI (@HelloSurgeAI) Oct 6, 2026

What is HANDBOOK.md - arXiv:2607.25398?

HANDBOOK.md measures whether LLM agents follow long, binding company policy documents (20-124 page SOPs) while performing routine professional tool-use tasks in mock company environments.

Enterprise work mostly
“instructs it to carry out routine professional work governed by an expert-written standard operating procedure of 20-124 pages”
arxiv.org/abs/2607.25398 ↗
Finance
“Tasks span five domains (finance, medical billing, insurance, logistics, and HR) and 10 fictional companies.”
arxiv.org/abs/2607.25398 ↗
Health and bio
“HANDBOOK.md has 65 agentic tasks across finance, medical billing, insurance, logistics, and HR”
surgehq.ai/research ↗
General capability
“HANDBOOK.md: A Benchmark for Long-Context Agentic Instruction Following”
arxiv.org/abs/2607.25398 ↗

Paper: arxiv.org/abs/2607.25398 ↗

Tier C

Niche: a handful of models scored, or built by an RL vendor.

How much stands behind a tier: usage, who backs it, a confirmed line from its own site or paper, and a clear read on X. High with three of the four, medium with two, low otherwise.

built by Surge AI
in curated RL directories (2 pts)

How we got to tier C

Adoption: built by Surge AI. Tier C.
Not yet read on X.
In curated RL directories: 2 points, a floor of C.
Confirmed: a line from its own site or paper places it.

A clear view on X moves a tier one step up or down; most entries don't move. How tiers work.

Where HANDBOOK.md - arXiv:2607.25398 sits

Its realm in the universe, live. Hover a star for its name; click to open it. Open it in the universe

The galaxy could not load. See every vendor on the board.

More Enterprise Workflows AI benchmarks

Enterprise Workflows RL environments

Next: browse all benchmarks. Entry from the RL Research daily scrape of public sources, 2026-10-11 snapshot.