HiL-Bench B

Name: HiL-Bench
Creator: Scale AI
Keywords: Coding,Computer Use,Enterprise Workflows,Long-Horizon

AI benchmark · Coding

Publisher: Scale AI
Domains: Coding Computer Use Enterprise Workflows Long-Horizon
On X: @scale_ai ↗
Tier: B why
Snapshot: October 11, 2026

Open benchmark ↗labs.scale.com/papers

Full listing name: HiL-Bench (2026-04-13)

Latest from Scale AI on X

@scale_ai ↗

Every RL environment we build runs on AgentEnv. Today we’re open-sourcing it, so more people can build the worlds agents learn in.
— scale_ai (@scale_ai) Oct 6, 2026
We’re excited to bring @nvidia's Open Agent Safety Platform technologies to Scale GenAI Portfolio, helping us build, train, deploy, run, and evaluate AI agents in controlled environments with enforceable security and a clear record of their actions.
— scale_ai (@scale_ai) Sep 28, 2026
We’re partnering with @googlecloud to make it easier for enterprises to put AI to work. Our joint reference architecture is a blueprint bringing the Scale GenAI Portfolio to Google Cloud, with Gemini Enterprise integration. Excited to give builders a proven foundation to put AI to work faster.
— scale_ai (@scale_ai) Sep 24, 2026

What is HiL-Bench?

HiL-Bench measures agents' help-seeking judgment (selective escalation): detecting missing, ambiguous or contradictory information in software engineering and text-to-SQL tasks and asking targeted questions via an ask_human() tool.

Coding mostly
“HiL-Bench includes two domains: software engineering and text-to-SQL.”
labs.scale.com/leaderboard/hil ↗
General capability
“HiL-Bench (Human-in-Loop Benchmark) measures help-seeking judgment in agents”
labs.scale.com/leaderboard/hil ↗

Who built HiL-Bench

Scale AI built it
“We design HiL-Bench to reflect this; tasks in this benchmark require agents to ask the right questions for full clarity”
labs.scale.com/leaderboard/hil ↗

Tier B

Established: 15 or more models scored, or curated RL directories rate it.

How much stands behind a tier: usage, who backs it, a confirmed line from its own site or paper, and a clear read on X. High with three of the four, medium with two, low otherwise.

10 models scored
built by Scale AI
in curated RL directories (4 pts)

How we got to tier B

Adoption: 10 models scored; built by Scale AI. Tier C.
Not yet read on X.
In curated RL directories: 4 points, a floor of B. Raised to B.
Confirmed: a line from its own site or paper places it.

A clear view on X moves a tier one step up or down; most entries don't move. How tiers work.

Where HiL-Bench sits

Its realm in the universe, live. Hover a star for its name; click to open it. Open it in the universe

The galaxy could not load. See every vendor on the board.

More Coding AI benchmarks

Coding RL environments

Next: browse all benchmarks. Entry from the RL Research daily scrape of public sources, 2026-10-11 snapshot.