Seven papers in one week generated their own RL environments: from 341 computer-use worlds to 64,900 work environments

Seven arXiv papers posted between October 12 and 19, 2026 trained agents in RL environments their authors generated by program and model instead of collecting from people. The scale runs from 341 long-horizon computer-use worlds in Agent World Model in the Wild to 64,900 synthesized work environments in Cowork-Gym.

The recipes converge on three design choices. Write a hidden specification of the world (or the reference solution) before generating anything the agent sees. Generate the verifier together with the task, and keep both only if they pass execution checks. Repair failing environments in a loop instead of discarding them.

The reported gains are large on the authors’ chosen benchmarks. WorkRover lifts its base model 26.27 points on JobBench. In MAI-Gym, RL in a mocked train-ticketing app raised success on the real app, on a physical phone, from 28.8% to 48.1%. Every paper measures against its own base model, not against expert-written tasks.

Key Takeaways

• Spec first: Tev and Data World build a hidden world, and Prime Intellect writes the reference solution, before any agent-facing file exists.
• Verifier with the task: generated checks are kept only after execution shows they pass good solutions and fail bad ones.
• Open stacks: Agent World Model in the Wild generates its environments with open-source models only.

What did each paper build?

One line per paper, in posting order:

• Davinci-Tool (Oct 12): 30,395 verified tool-use trajectories across 2,260 tool names, with restorable states and executable verifiers. Training is SFT then DPO (preference training), not RL.
• Cowork-Gym (Oct 13): 64,900 environments across 14 work scenarios, with a stateful simulated user who replies and edits the shared workspace.
• WorkRover (Oct 13): environments that evolve; when a task breaks, the system tests combinations of revised work context, task and verifier, and keeps the smallest repair that passes.
• Tev (Oct 13): 1,170 verified long-horizon environments across 7 domains, each rendered from a hidden specification that also defines progress.
• MAI-Gym (Oct 14): a GUI-agent platform that runs 20,000 parallel environment instances, generates resettable mocked apps, and releases 2,000 verified tasks.
• Data World (Oct 15): 2,135 data-analysis training tasks from 1,037 separate generated worlds, graded by rubrics.
• Agent World Model in the Wild (Oct 19): 341 environment-task pairs spanning GUI, MCP tools and the command line, some needing more than 100 steps; GPT-6 Astra solves 60%.

Figure: Zou et al., Cowork-Gym, arXiv 2610.17670, Figure 1 (cropped), CC BY 4.0 · source

What do the recipes have in common?

Three choices recur, each aimed at a known way generated tasks fail:

1. Write the world before the files. Tev constructs a latent world, a complete hidden specification of the domain, the workflow and the expected state after each step, then renders tasks from it. Data World states the principle directly: “specify the world behind the files before generating the files themselves.” Prime Intellect’s Nemotron tasks, published October 15, are built solution-first, with the checks derived from what the reference solution changes. A hidden specification gives the verifier something true to check against, which a task generated file by file lacks.
2. Gate the verifier by execution. Agent World Model in the Wild keeps a task only if it passes rollouts from different agents, agreement between independent judges and counterfactual probes of the judge. MAI-Gym ships only execution-validated verifiers. This is the defect class SWE-Bench Pro V2 found in 247 human-built tasks: a verifier that grades the wrong thing.
3. Repair instead of regenerate. Cowork-Gym runs a build, execute and revise loop on each environment. WorkRover goes further and updates the task, its context and its verifier together, so a fix to one does not silently break the others.

Two papers also densify the reward. Tev turns its hidden progress metric into a per-step reward, which it says makes otherwise intractable long tasks trainable, and WorkRover credits earlier turns that later checks depended on.

Figure: Hu et al., Tev, arXiv 2610.18868, Figure 3, CC BY 4.0 · source

Do skills learned in generated worlds transfer?

The MAI-Gym phone result is the cleanest evidence in the set: the training app was a mock, and the test ran on the real app on a physical device. Agent World Model in the Wild reports gains on 8 out-of-distribution benchmarks for web, claw and computer-use agents. Tev trains only on tasks related to Agents’ Last Exam and still improves on Terminal-Bench 3.0 and 4.0 subsets.

The limits are shared. At least five of the seven train Qwen models, all seven report gains over their own base, and none of the abstracts compares generated tasks with the same number of expert-written ones, the comparison a buyer of expert data would want.

What this means

Generation now has a recognizable method, and it is the same method environment vendors sell: a specified world, a verifier proven by execution, and a repair loop. What expert authoring still supplies is the specification itself, the part ground truth depends on. Paper Instruments’ Ultramarine showed a full product built this way; these papers show the recipe spreading to open research groups.

FAQ

What is a synthetic RL environment?

A training environment generated by programs and models instead of authored by people: the apps, files and records the agent works in, the task, and the check that decides the reward. The papers above generate all three and keep an environment only after executed checks show it is solvable and correctly graded.

Can an agent trained only in generated environments work on real apps?

MAI-Gym’s result says it can transfer: RL in a mocked train-ticketing app raised task success on the real app, on a physical phone, from 28.8% to 48.1%. Agent World Model in the Wild reports gains on 8 out-of-distribution benchmarks after training only in its generated worlds.

What is dense progress reward?

A reward paid at each step for progress toward the goal, instead of once at the end. Tev measures progress against the hidden specification it generated each task from and converts it into a per-step reward, which helps on long tasks where a single final reward is too sparse to learn from.