A production-grade RL environment spec freezes state, reward, and splits before code

Of the roughly 99 tracked environments, we found none whose public documentation publishes verifier (the program that grades each episode) calibration numbers together with a checksum-frozen corpus (our reading of their documentation, not a vendor survey). The specs labs are likely to pay for, in our judgment, fix state machine, tool schemas, reward, curriculum, and splits before any simulator code exists: a 572-line reference contract we examined does this, backed by 79 checksum-frozen case assets, down to a 12,840-row inventory subledger and an oracle script (a reference solution) per case. A spec of this kind precedes any running system: it is the contract a build is checked against, and every field carries a provenance label so a reader can tell a validated capability from a planned one. The contrast with the market is direct: Surge AI’s CoreCraft builds large, messy enterprise worlds. Mess is valuable, but a verifier can only be calibrated against episodes that replay byte for byte.

Key Takeaways

• The reference contract fixes six design decisions and 79 SHA-256-frozen case assets before any simulator code exists.
• RL’s reproducibility problem is documented back to 2017, and a verifier can only be calibrated against episodes that replay byte-identically.
• Our read: the two market philosophies, frozen contracts and large messy enterprise worlds, converge on worlds that are messy but deterministic from a seed.

What does a production-grade spec contain?

The reference contract fixes six things before any simulator code is written:

• the episode state machine
• tool schemas
• the reward function
• curriculum
• split policy
• anti-gaming requirements

Every episode follows a state machine from initialization through investigation to submission, with terminal conditions for safety failures, integrity violations, and budget exhaustion. That architecture matches what the strongest published agent benchmarks converged on: τ-bench computes reward by comparing the final database state against the annotated goal state, the same discipline the contract encodes: fix the state machine first, grade the end state.

The contract also freezes 79 case assets across its reference builds: public performer assets, private evaluator assets, and an oracle script per case. The assets include:

• schematics and sensor traces
• a 12,840-row inventory subledger
• policies and private truth files
• deterministic tool responses
• authored verifier request/expected-result pairs

Every file has a SHA-256 digest. The corpus is frozen before a single episode is served.

Why does the spec fix determinism and splits first?

Because published deep-RL results have failed to reproduce. Henderson et al. (2017) showed that seeds, implementation details, and uncontrolled nondeterminism swamp reported deep-RL results. The generalization work behind CoinRun and Procgen demonstrated that agents overfit surprisingly large sets of training levels, which is why the contract versions and freezes the generator and the split policy rather than individual instances.

The spec’s answer: splits are fixed in the contract, test families are held out by construction, and every episode replays byte-identically from a seed and version. If the environment cannot do that, the verifier has no stable ground truth to be calibrated against.

Chart: Surge AI, February 19, 2026 · source

What is the market counterpoint?

Surge AI’s EnterpriseBench CoreCraft builds large, messy enterprise worlds: synthetic organizations with 2,500-plus entities and 23 tools exposed over the Model Context Protocol (MCP). In our reading it argues that realism beats determinism.

Mess is valuable. But if two runs of the same episode produce different states, the verifier cannot distinguish a correct agent from a lucky one. Our read is that the two philosophies converge on frozen chaos: environments that are complex and messy but deterministic from a seed, so every episode replays byte-identically and the verifier’s ground truth is stable.

How does the spec label what is not built yet?

The reference contract labels every field with its provenance:

• observed
• inferred
• author-proposed
• expert-validated
• measured in a pilot

Readiness gates are named explicitly, so a reader can tell a validated capability from a planned one.

The Agentic Benchmark Checklist audit found that task-validity and outcome-validity flaws in popular agentic benchmarks distort measured performance by up to 100% relative (Zhu et al., 2025). Named approval decisions and hard dollar ceilings per validation run, with automatic stops, are part of the contract. The acceptance criteria specify deterministic seed counts, generation-yield floors, and 100% byte-identical replay under the same version and seed. A spec passes these gates before it becomes a system a lab would pay for.

What this means

A spec that fixes state, reward, and splits before code, freezes its corpus with checksums, and labels its unbuilt parts is the minimum bar rlresearch.ai’s reference designs set before a build starts. The calibration numbers it must then publish are the next bar.

FAQ

What is a checksum-frozen asset?

A checksum-frozen asset is a case file (schematic, sensor trace, subledger, oracle script) whose SHA-256 digest is recorded in a manifest, so any change to the file invalidates the corpus and the same seed and version always replay the same episode.

Why does determinism matter so much?

Verifier calibration compares the verifier’s scores against expert-adjudicated ground truth on identical episodes. If the environment cannot replay an episode byte-identically, there is no stable ground truth to calibrate against, and every measurement is of a slightly different episode.

What is CoreCraft?

CoreCraft is Surge AI’s EnterpriseBench: a synthetic PC-retailer support organization with 2,500-plus entities and 23 MCP tools, scored against expert-authored rubrics used as the reward signal. It builds large, messy enterprise worlds; the counterpoint is that mess without deterministic replay cannot calibrate a verifier.