Research and Analysis on RL Research

Laya vs Jev: Did Decision AI Arrive a Year Earlier?

Sun, 20 Sep 2026 00:00:00 +0000

Laya’s creator published related reinforcement learning (RL) research before TypeSafe launched Jev; the evidence supports an earlier precursor, with important differences from Jev. The current Laya release, reviewed September 20, 2026, offers 3 open-weight checkpoints, meaning developers can download and modify the learned model parameters. In his retrospective, ConvAI Innovations founder Nandakishor Mukkunnoth argues that his earlier work anticipated Jev’s approach: use a model to make structured decisions without generating prose. His March 2025 paper does document reinforcement learning for sales-conversion probabilities. It does not establish an identical architecture, and the present Laya model page was created after Jev’s launch. For engineers, the useful distinction is access: Laya exposes code and weights that can be inspected and trained. Comparing the systems still requires matching tasks, training exposure, and deployment conditions. The published comparison charts combine separate studies, so they cannot settle which model is better for a particular application.

Is TypeSafe's Jev an LLM? What changes for RL environments

Sat, 19 Sep 2026 00:00:00 +0000

TypeSafe says Jev is not a large language model (LLM); in reinforcement learning (RL) environments, it can select actions but does not directly generate text or code. Its documentation, checked September 19, 2026, defines 3 decision types: Choice, Score, and Noul, for selecting an option, rating a scale, and estimating whether a statement is true. Application code turns those answers into actions. That makes Jev a candidate for routing requests, choosing tools, or deciding when to escalate. It cannot replace the generative component of a coding or writing task. An RL environment presents observations, accepts actions, and rewards their consequences. It can therefore evaluate either kind of model, provided the task fits. The useful comparison is whether each system completes the same work with less cost or delay, using the same evidence and success criteria.

Anthropic's AI agents beat 28 human experts at alignment research, and humans became the baseline

Tue, 15 Sep 2026 00:00:00 +0000

Automation has reached the last human stage of the AI training loop: Anthropic’s automated alignment researchers, agents that run the safety-research loop end to end, beat proposals from 28 experienced AI safety researchers on all seven alignment failures where human ideas were collected. Anthropic published the study on August 28, 2026, and on September 15 Surge AI, the expert data vendor that ran its human baseline, described that work. Alignment research is the work of finding methods that stop models from deceiving, flattering, or gaming their graders. Across ten alignment failures (ways a model’s behavior departs from what its developers intend), including deception, sycophancy, jailbreaks, privacy violations, and reward hacking, the agents found methods that improved safety benchmarks without hurting general capability. On September 9 this site wrote that one stage still needs people, deciding what counts as correct. This study moves that boundary: proposing and testing fixes got automated, and humans still choose which failures matter, build the benchmarks, and judge whether the outcomes count as better.

1,700 RL tasks lifted Kimi K2.7 on every coding benchmark tested

Sat, 12 Sep 2026 00:00:00 +0000

Broad expert task portfolios, not benchmark targeting, are now the reliable RL training recipe. On September 11, 2026, Surge AI, the expert data vendor, published a study in which 1,700 agentic coding tasks lifted Kimi K2.7, Moonshot AI’s open-weight model, on all five external coding benchmarks tested, by 12.5 percentage points on average, with gains from +4.7 to +20.0 depending on the benchmark. The training was reinforcement learning only, with no supervised warmup (no phase of copying example solutions first), on 1,000 repository-style fixes and 700 terminal jobs. The tasks came from Surge AI’s existing catalog, were not selected to target any of the five benchmarks, and were collected before three of the five existed. That independence is the finding, and it matches the 397B (397 billion parameter) result Mercor published on September 1 and the office-work transfer in August: buy a broad portfolio of expert tasks with calibrated grading, train against it, and the gains show up on evaluations nobody optimized for.

AI agents solved a Millennium Prize problem. A mathematician says they raced him to it

Thu, 10 Sep 2026 00:00:00 +0000

On September 8, 2026, OpenAI announced a solution to the Navier-Stokes Millennium Prize problem, and by OpenAI’s own figures the proof took around 10,000 coordinating AI agents 88 hours. When a lab can point that many agents at one problem, unpublished expert work becomes training input, and provenance (the record of who produced a piece of expert work, and under what agreement) becomes the priced feature of expert data. Levent Alpöge, a Harvard mathematician working on the same problem, stated publicly that he had contacted OpenAI on September 2 because he believed the company had learned about his private work and had assembled a group to compete with him. Sebastien Bubeck of OpenAI replied that nothing was locked and that the company had been willing to talk. The same night, a pretraining researcher resigned from Anthropic over the race itself. The training loop this site tracks (models learning from scored attempts at expert-built tasks) just consumed its first Millennium-grade expert problem, and the fight is about who owned the inputs.

An Anthropic researcher resigned over self-improving AI: human grading is the one loop stage that does not scale

Wed, 09 Sep 2026 00:00:00 +0000

Human grading is the one stage of the AI training loop that does not scale with compute, and the race Jacob Coxon resigned over on September 8, 2026 is a race to automate the rest of it. The race is visible in rlresearch.ai model score tracking: after a September 9 refresh that changed 172 model scores, 0.25 points separate the top two models, 84.36 against 84.11. Coxon quit Anthropic and wrote that his employer and OpenAI, where he says he spent a combined three years, are racing to build self-improving superintelligence and taking a reckless gamble with human lives; wire services picked it up within hours. Today’s models improve through a simple loop: give the model real work with a checkable answer, score the attempt, adjust the model toward what scored well, repeat millions of times. Self-improvement means the model starts running parts of its own loop. One stage still needs people: deciding what counts as a correct answer, and it does not get cheaper as compute grows.

AI agents learned to hedge. The benchmark patched it and published the cost

Tue, 08 Sep 2026 00:00:00 +0000

When AI agents learn to game a benchmark, the lasting fix is verifier calibration (measuring how often the grader errs in each direction and adjusting it), and on September 8, 2026, Mercor, the expert data vendor, published what that calibration costs: the judge in its rebuilt APEX-Agents knowledge-work benchmark now rejects more correct answers, a false-negative rate of 8.0% up from 5.3%. The gamed behavior is hedging: instead of committing to one answer, a model submits several and hopes the grader credits the right one. Mercor calls this scattergunning, and it rebuilt APEX-Agents to score every hedged answer as zero. The 1.1 release also audits tasks with experts and instructs models not to hedge. The patch had a measured cost, more correct answers now get rejected, and Mercor published the number anyway. This matches what our environment-quality research found about trustworthy verification: when a grader gets gamed, the maintainer recalibrates it, accepts a measured cost in one direction to close a hole in the other, and publishes the numbers.

RL environments just moved a 397B model. The training guide is public

Wed, 02 Sep 2026 00:00:00 +0000

The clearest proof yet that RL environments (packaged work tasks, each with a checkable answer and a grader, that a model trains against) move frontier-scale models is now public. In a training guide published September 1, 2026, Mercor, the expert data vendor, reports reinforcement-learning post-training of Alibaba’s Qwen3.5-397B-A17B on 1,928 expert-created knowledge-work tasks, lifting Pass@1, the share of tasks solved on the first try, from 16.11% to 27.29% on a held-out benchmark of 480 realistic tasks. That is a 70 percent relative improvement at 397B-parameter scale, with no supervised warmup (no phase of copying example solutions first), using the open-source SkyRL framework. The 1,928 tasks are sold as a cataloged dataset. Our June census argued the field over-invests in measuring capability and under-invests in the environments that produce it. This guide shows the production side working: a vendor sold expert tasks, a frontier-scale model trained on them and improved, and the vendor published the method.

How stable are AI leaderboards? One refresh moved 135 model scores

Sat, 29 Aug 2026 00:00:00 +0000

Most single-day movement on AI leaderboards is the scorer, not the models. In the August 29, 2026 refresh of our model score tracking, 135 models had their composite scores (one number aggregating many benchmark results) updated at once, and 129 of them drifted upward together by at most 0.22 points. The drifting models came from 29 different makers: Google, xAI, Alibaba, Anthropic, Moonshot, and two dozen more, all nudged the same direction on the same morning. Twenty-nine competitors do not ship improvements on the same morning; a change to the scoring method moves them all at once. One real story hid inside the same refresh: a single model, Qwen3.8-Flash-Next, fell 6.22 points while nothing else moved more than 0.22 in either direction. Reading a leaderboard means telling those two movements apart: the scoring method shifting under everyone, versus one model being re-measured. The difference is invisible if you only quote one model’s number.

Who defines ground truth for AI agents?

Mon, 24 Aug 2026 00:00:00 +0000

Human experts define ground truth for AI agents: authors create private truth, reviewers adjudicate it, and models never approve it. The bars rlresearch.ai’s reference spec proposes a trusted environment publish are verifier (the program that grades each episode) precision and recall at or above 0.95, hard gates at exactly 1.00, and mean absolute reward error measured against adjudicated expert scores. The pattern in rlresearch.ai’s reference environment specs is three roles with hard access boundaries. Authors create cases and private truth. Performers complete work in a clean VM seeing only public assets. Reviewers adjudicate correctness but cannot change a score without a reason code, a logged justification category. Models draft, generate variants, and red-team, meaning they attempt to break the environment; approval and release promotion stay human. The thresholds are the reference spec’s proposed bars, not measured industry norms. This verification work is what Mercor’s 27% gross margin (its 2025 figure) pays for.

Why AI agent sandboxes run in microVMs with zero inbound ports

Mon, 17 Aug 2026 00:00:00 +0000

Long-horizon episodes need the operating system and applications the work runs on, plus hard isolation, and the pattern rlresearch.ai’s reference infrastructure spec adopts is microVMs with zero inbound ports. A microVM is a lightweight virtual machine with a minimal device model that boots in milliseconds while keeping a hardware-virtualization boundary between the agent and the host. Firecracker, the microVM monitor built for AWS Lambda (Agache et al., NSDI 2020), runs Linux sessions; Cloud Hypervisor, the open-source virtual machine monitor hosted by the Linux Foundation, runs Windows Server for enterprise work that is Windows-native. All traffic flows over outbound tunnels, gateway URLs are single-session and expiring, and capacity is reserved in a database before any VM exists. Two problems remain open in every public environment stack we have examined: per-episode application-database reset for deterministic verifiers, and an action-level GUI event schema for turning desktop sessions into training data.

How AI agents cheat their training environments

Mon, 10 Aug 2026 00:00:00 +0000

Under RL pressure, agents game environments in predictable ways, and the counter in rlresearch.ai’s reference designs is an integrity gate: a check that runs before scoring and carries zero reward weight. The evidence is quantitative: on ImpossibleBench (Zhong et al., October 2025), where tasks are unsolvable so any pass proves test exploitation, frontier models exploited test cases in up to 76% of tasks in one variant. The formal name for the gaming is reward hacking: optimizing a proxy for the objective in ways that degrade the objective itself (Skalse et al., 2022). Across the environment designs rlresearch.ai has reviewed, the gaming moves range from format masquerading as competence to memorization of leaked test structure, and the failures start with reward design: a multi-component weighted rubric leaks reward through prose formatting and keyword compliance. The fix in rlresearch.ai’s reference designs is one verifiable outcome bit, backed by integrity gates that zero the episode’s reward on a fabricated citation or a prohibited action.

Training AI on office work made it better at coding

Mon, 03 Aug 2026 00:00:00 +0000

Environment training transfers across domains. On August 3, 2026, Surge AI, the data-labeling and evaluation vendor, reported that a model (Qwen3.5-122B-A10B) post-trained on office tasks with no coding in the mix gained 5.8 points on SWE-Bench Pro, a coding benchmark the model was never trained on; Surge published the delta, not the baseline score. The post-training mix was documents, spreadsheets, web research, planning, and tool use across office services. Surge reads the gain as transferable goal-directed execution: planning, multi-step execution, error recovery, and producing a verifiable output. The result is vendor-reported and not yet independently replicated. If it holds, it changes what labs are buying: an environment’s value is measured by the execution it demands rather than by domain overlap with the target benchmark. AfterQuery reports in-domain expert-data uplifts on τ²-Bench, not cross-domain transfer, and transfer raises the stakes on contamination controls, the checks that a training environment shares no structure with the evaluation benchmark.

Mercor's APEX-Accounting benchmark measures month-end close and leaves audit out of scope

Fri, 31 Jul 2026 00:00:00 +0000

Mercor and Ramp released APEX-Accounting on July 31, 2026: a benchmark for month-end close and bookkeeping workflows with 160 held-out (unpublished) tasks across 10 simulated companies, expert-authored rubrics averaging 13.7 criteria per task, and an open-source AI judge that Mercor reports at 97% agreement with expert graders. A benchmark of this kind cannot carry evidence integrity, which is what audit work is graded on, and Mercor’s own scope note says as much: APEX-Accounting does not evaluate tax, audit, consolidation, or external reporting. A reference audit case we examined makes the distinction concrete: a 12,840-row inventory subledger totaling $8.305 million, in which the graded competency is whether each entry traces to a source document and unsupported entries are escalated. Claude Fable 5 tops the APEX-Accounting leaderboard at 56.4%, and the most consistent model solved 2.6% of tasks correctly in all eight runs. Accounting tasks were among the 96.5% without definitive benchmark coverage in June; the close-and-bookkeeping part of that gap is closing, and the evidence-integrity part is not.

A production-grade RL environment spec freezes state, reward, and splits before code

Mon, 20 Jul 2026 00:00:00 +0000

Of the roughly 99 tracked environments, we found none whose public documentation publishes verifier (the program that grades each episode) calibration numbers together with a checksum-frozen corpus (our reading of their documentation, not a vendor survey). The specs labs are likely to pay for, in our judgment, fix state machine, tool schemas, reward, curriculum, and splits before any simulator code exists: a 572-line reference contract we examined does this, backed by 79 checksum-frozen case assets, down to a 12,840-row inventory subledger and an oracle script (a reference solution) per case. A spec of this kind precedes any running system: it is the contract a build is checked against, and every field carries a provenance label so a reader can tell a validated capability from a planned one. The contrast with the market is direct: Surge AI’s CoreCraft builds large, messy enterprise worlds. Mess is valuable, but a verifier can only be calibrated against episodes that replay byte for byte.

Which jobs can become RL environments?

Fri, 17 Jul 2026 00:00:00 +0000

Jobs whose core work happens on a computer and produces a checkable output can become RL environments; jobs with a critical physical dependency cannot, at any budget. rlresearch.ai’s screening framework scores each job on ten dimensions, from computer-work share to output verifiability, with one hard rule: a critical physical dependency vetoes construction outright, with no score adjustment. A job that requires hands-on equipment operation, physical inventory observation, or in-person client interaction cannot become a sandbox environment, however well it scores everywhere else. Three reference workflows screened with the framework, failure diagnosis, engineering test analysis, and audit workpaper completion, pass the gate with computer-work shares of 0.82 to 0.96, and each names the physical capabilities it excludes. Surge AI’s Chartography, launched July 16, 2026, passes every screen. The framework sorts the 66 tracked occupations without environments into vetoed and not yet built.

Why Mercor is buying Deeptune

Fri, 10 Jul 2026 00:00:00 +0000

Mercor’s move to acquire Deeptune says the constraint has shifted from expert networks to the environments themselves, and our occupation data shows the gap the deal targets: 0 of 66 tracked occupations have a corresponding RL environment. On July 9, 2026, Mercor, the AI expert marketplace, announced it would acquire Deeptune, an RL environment vendor that had raised $43M from Andreessen Horowitz (a16z) in March 2026 and spent two years recreating hundreds of enterprise applications, from spreadsheets to Salesforce, as agent simulations. Mercor’s announcement says the constraint has shifted to the environments themselves. Without recreated enterprise applications, experts can write rubrics and grade outputs but cannot train agents to navigate real workflows. Our vendor census shows the same lopsidedness: coding is crowded, math has two sellers, and medical has one.

3,029 AI benchmarks, 99 RL environments: the field is over-indexed on evaluation

Fri, 03 Jul 2026 00:00:00 +0000

The benchmark pile keeps growing while cataloged RL environments are rare. Our catalog counts 3,029 unique benchmarks and 24,810 published model scores against 99 cataloged RL environments, a 31-to-1 ratio. An RL environment is an executable task world that returns a verifiable reward; a benchmark is a fixed dataset with a scoring script. Of the 3,029 benchmarks, 535 have enough structure, documentation, and model coverage to appear in the public catalog view. Expert verification is what moves model numbers. In the week of June 29, 2026, Surge AI trained a 4B model on 1,000 prompts graded by expert-written rubrics to parity with a model 60 times larger, and AfterQuery helped NVIDIA lift Nemotron 3 Ultra from 35.3 to 46.7 on GDPval, OpenAI’s benchmark of professional work. The field has produced 31 benchmarks for every cataloged environment, the asset that changes what models can do.

AI benchmarks cover only 3.5% of real work

Fri, 26 Jun 2026 00:00:00 +0000

Our study joining 202 O*NET occupational tasks, from the US Department of Labor’s occupational database, against 223 candidate benchmarks with an LLM judge (a language model scoring each task-benchmark pair against a rubric) finds definitive coverage of 3.5%. Only 7 of 202 tasks are definitively covered by an existing benchmark. The study cost $13.79 across 441 judge calls, and a robustness check, a recall audit of 4,935 task-benchmark pairs against every unlinked benchmark, found zero missed matches for $8.82, bringing the combined cost to $22.61. Benchmark success materially overstates workflow competence: existing benchmarks test isolated calculations or document operations rather than the multi-step occupational workflows they claim to represent. Vendor benchmarks show the same gap from the other side: every frontier model scored below 25% strict pass@1 on Surge AI’s HANDBOOK.md (June 25, 2026), and frontier agents completed under 25% of Mercor’s APEX-Agents tasks (January 2026).

What do AI expert job postings reveal about RL environment demand?

Mon, 15 Jun 2026 00:00:00 +0000

Expert job listings on Mercor, AfterQuery, and micro1 are a leading indicator of where AI labs are investing in environment development. We track active expert postings across all three platforms, deduplicated so cross-posted roles count once, and classify each by domain using the platform’s own taxonomy first, with title inference only as a fallback. The pattern is consistent: labs recruit domain experts months before the environments those experts build reach a training run. Recruiting concentrates in a few domains (coding, finance, legal, medical), while our vendor census counts 21 of 38 tracked companies selling coding environments, 18 selling enterprise workflows, and 18 selling computer-use environments; math has 2 sellers and medical effectively one. Where recruiting volume diverges from what vendors currently sell is where the next environment categories will appear.

The RL environment industry is 38 companies, mostly under 50 people

Wed, 10 Jun 2026 00:00:00 +0000

Our census of 38 tracked RL environment vendors finds 31 with 50 or fewer employees. The companies supplying the most capitalized labs on earth are a cottage industry: Scale AI and Turing are the only firms above 200 staff, and everyone else, including the vendors whose benchmarks appear in frontier system cards (the documents labs publish with each model release), runs on a small team. On June 8, 2026, AfterQuery, a 51-200-person company from Y Combinator’s Winter 2025 batch, published a +21.4% net win-loss gain (wins minus losses as a share of pairwise comparisons) on GDPval, OpenAI’s benchmark of professional work tasks, using on-policy distillation, a method that trains a student model on its own outputs with a teacher model’s feedback. The modal band is 11 to 50 employees, and both of the two largest vendors are legacy data-labeling companies that expanded into environments rather than companies founded to build them.

Why RL environment vendors earn services margins and software valuations

Tue, 09 Jun 2026 00:00:00 +0000

Frontier labs are shifting spend from labeled data to executable environments, and the vendor margins show where the leverage sits. Mercor, the expert marketplace that pays contracted professionals to build and grade training data for AI labs, runs roughly $2B in gross annualized revenue at a 27% gross margin, and is valued at $10B after a $350M Series C. AfterQuery closed a $30M Series A at a $300M valuation and, in the same April 9, 2026 announcement, reported passing a $100M revenue run rate. Mercor’s margin profile looks nothing like software: software gross margins typically run 70 to 80%, and a 27% margin is the cost of paying an expert for each unit of graded work. The category winner sells expert labor organized around verification, and the labs buying it cite vendor evals in their own system cards, the documents labs publish with each model release (June 2026). The money is moving from annotation budgets to environment budgets, toward the companies that can grade what a model does.