RL environments and benchmarks for computer use agents

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

Covers computer use“Evaluates computer-use agents on 108 long-horizon workflows.” source ↗

Sells RL environments“Snorkel helps frontier labs and AI teams develop specialized training data and environments that set their models and agents apart.” source ↗

Covers computer use“Human-demonstrated interactions across browser and desktop environments — teaching models to navigate and operate software end-to-end.” source ↗

Sells RL environments“Provides custom RL environments built on top of real APIs, MCP servers, and developer tools” source ↗

Covers computer use“teaching them to use computers, navigate the web, and reason for the first time.” source ↗

Sells RL environments“We create rich, complex RL environments that challenge agentic models in novel ways, and design verifiers that reward their behavior.” source ↗

Sells RL environments“an applied research lab dedicated to the scale out of environments - diverse and high-fidelity training gyms that accurately model the real world” source ↗

Sells RL environments“We build training gyms for frontier research labs.” source ↗

Covers computer use“AI sandbox environments for training and evaluating computer-use agents on realistic simulated websites.” source ↗

Sells RL environments“Halluminate is a data research lab building benchmarks and RL environments to push the frontier of AI in knowledge work.” source ↗

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

Covers computer use“Software engineering, computer use, and long-horizon reasoning” source ↗

Sells RL environments“Every simulation lab is a self-contained world where your agent operates, complete with the users, tools, data, and tasks it will face in production.” source ↗

Covers computer use“Collaborative reinforcement learning environments for computer use research.” source ↗

Sells RL environments“Deterministic, pixel-perfect reinforcement learning environments with frame-accurate state control” source ↗

Covers computer use“Full desktop and browser control, judged on the end state of long, multi-step tasks.” source ↗

Sells RL environments“We build **RL environments** for frontier AI labs” source ↗

Covers computer use“UiPath brought its UI-CUBE enterprise computer-use benchmark onto HUD” source ↗

Sells RL environments“The platform for building RLenvironments” source ↗

Covers computer use“High-fidelity clones of full software suites, wiring together multiple connected applications, from EHRs to enterprise apps and beyond.” source ↗

Sells RL environments“Refresh builds simulated worlds for coding and computer use, with verifiable rewards that move models forward.” source ↗

Sells RL environments“BenchFlow builds the environments AI agents learn in” source ↗

Covers computer use“Train and evaluate agents on real computer-use tasks.” source ↗

Sells RL environments“Cua has 4K+ environments and 400K+ tasks spanning desktop apps, cross-app workflows, browser surfaces, and generated state variations.” source ↗

Covers computer use“Data + environments for training computer-use AI.” source ↗

Sells RL environments“End-to-end CAD environments with the full prompt, reference inputs, complete screen recording, deliverable previews, and step-level action trajectories.” source ↗

Covers computer use“the agent operates the VM itself: pixels, keyboard, shell, files.” source ↗

Sells RL environments“Plato is an applied research lab that transforms real-world data into simulated environments for training and evaluating agents.” source ↗

Sells RL environments“Interactive environments where agents take on open-ended tasks and make real decisions, with verifiers built to reward the behavior that matters.” source ↗

Sells RL environments“Adapt the commit into a task description, golden patch (the solution), and test patch (verification tests).” source ↗

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

Tests computer use“a benchmark of 108 long-horizon, real-world computer-use workflows where even frontier AI agents complete only 20.6% of tasks outright after 300+ steps each.” source ↗

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

Tests computer use“a benchmark of 108 long-horizon, real-world computer-use workflows where even frontier AI agents complete only 20.6% of tasks outright after 300+ steps each.” source ↗

Snorkel AI · built it with others“Senior SWE-bench built by Snorkel with Princeton University and UW-Madison” source ↗

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

Tests computer use“Computer-Use: A GUI-based environment simulating human-computer interaction (mouse, keyboard).” source ↗

Scale AI · built it“We design HiL-Bench to reflect this; tasks in this benchmark require agents to ask the right questions for full clarity” source ↗

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

BenchFlow · built it“We publish the environments and benchmarks we use to evaluate agents on real work — papers, datasets, and live sites. SkillsBench, ClawsBench, Robo Use” source ↗

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

Collinear · built it“Evaluations we build to find where models fail” source ↗

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

Tests computer use“Cua-Bench is a collection of desktop and mobile tasks with a harness for evaluation and training to help agent makers quantify their agents' computer-use mastery.” source ↗

Cua · built it“built with love by cua.ai” source ↗

Tests computer use“WebBench is an open, task-oriented benchmark designed to measure how effectively browser agents handle complex, realistic web workflows.” source ↗

Tests computer use“A 50-task benchmark for evaluating whether AI models can complete precise AutoCAD work using computer-use alone.” source ↗

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

Tests computer use“The large majority of tasks require both GUI interaction and MCP tool-use interleaved for successful completion” source ↗

Chakra Labs · built it“Chakra Labs has built a new benchmark to measure this capability directly: MAGI (MCP and GUI Benchmark).” source ↗

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

Scale AI · built it“Today we're introducing READY (Reliable Enterprise Agent Deployment), a suite of industry-specific benchmarks that measure agents the way enterprises actually use them” source ↗

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

Tests computer use“A benchmark for web agents to perform tasks on realistic websites.” source ↗

Tests computer use“BrowserBench exercises multiple hosted Chromium providers against a shared set of autonomous web-browsing tasks.” source ↗

AfterQuery · contributed tasks or data“end-to-end legal tasks drawn from real cases across 28 practice areas, with a public case co-created with AfterQuery.” source ↗

Collinear · built it“Evaluations we build to find where models fail” source ↗

Collinear · built it“To apply our TraitBasis method systematically, we developed τ-Trait, a new benchmark designed specifically to measure agent robustness.” source ↗

Andon Labs · built it“We're releasing Vending-Bench 2, a benchmark for measuring AI model performance on running a business over long time horizons.” source ↗

Andon Labs · built it“Blueprint-Bench 2 builds on our original paper with an agent-only evaluation, improved scoring, and a persistent notepad for cross-apartment learning.” source ↗

Andon Labs · built it“As part of our continued effort to understand AI’s impact on the physical world, we release Drone-Bench” source ↗

BenchFlow · built it“We publish the environments and benchmarks we use to evaluate agents on real work — papers, datasets, and live sites. SkillsBench, ClawsBench, Robo Use” source ↗

BenchFlow · built it“We publish the environments and benchmarks we use to evaluate agents on real work — papers, datasets, and live sites. SkillsBench, ClawsBench, Robo Use” source ↗

BenchFlow · built it“We publish the environments and benchmarks we use to evaluate agents on real work — papers, datasets, and live sites. SkillsBench, ClawsBench, Robo Use, PostTrain, and FrontierPhysics.” source ↗

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

Snorkel AI · contributed tasks or data“Snorkel AI served as Harvey’s data partner for BigLaw Bench: Research, bringing its expertise in creating complex expert data for frontier AI” source ↗

Tests computer use“our agents operate in the environment through code, structured tools, and mouse-driven GUI control.” source ↗

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

Tests computer use“"Mcp-Image": "hudevals/hud-remote-browser:0.1.3"” source ↗

Tests computer use“Frontier computer-use agents on 25 expert-authored KiCad schematic tasks.” source ↗

Tests computer use“Cua-Bench is our computer-use benchmark, published with Snorkel, live at cua.ai/cuabench. It tests agents against real GUI work” source ↗

Cua · built it with others“Cua-Bench is our computer-use benchmark, published with Snorkel, live at cua.ai/cuabench.” source ↗

Tests computer use“agents can operate freely within a real Windows OS and use the same wide range of applications, tools, and web browsers available to human users” source ↗

Tests computer use“Each brief defines what the agent receives, what it must change and what it must preserve.” source ↗

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

No line from its own site or paper places it in Computer Use yet; the catalog lists it here.

Tests computer use“agents operating in terminals are increasingly capable of performing a broader range of general computer-use tasks beyond coding” source ↗

Sells data“Deterministic environments with frame-accurate state control, high-fidelity trajectory datasets, and mixed-modality training capability.” source ↗

Covers computer use“Collaborative reinforcement learning environments for computer use research.” source ↗

Sells data“We run the rollouts on the same environments and deliver verified trajectory datasets, packaged for your data-ingestion pipeline” source ↗

Covers computer use“Train and evaluate agents on real computer-use tasks.” source ↗

Sells data“Rich computer-use data built around complete workflows, synchronized interaction signals, and verifiable outcomes.” source ↗

Covers computer use“Data + environments for training computer-use AI.” source ↗

Sells data“Draws on 300,000+ verified practitioners across medicine, law, finance, engineering, and more” source ↗

Covers computer use“Human-demonstrated interactions across browser and desktop environments — teaching models to navigate and operate software end-to-end.” source ↗

Sells data“We turn these worlds into post-training datasets and reinforcement learning environments” source ↗

Covers computer use“Navigating across apps, browsers, and operating systems.” source ↗

Sells data“500B+ tokens of agent training data generated” source ↗

Covers computer use“Software engineering, computer use, and long-horizon reasoning” source ↗

Sells data“A frontier human data and evaluation lab making AI better for creativity.” source ↗

Covers computer use“Computer-use trajectories from real-world projects with reasoning.” source ↗

Sells data“DataVendorConnect post-training suppliers with labs.” source ↗

Covers computer use“UiPath brought its UI-CUBE enterprise computer-use benchmark onto HUD” source ↗

Sells data“Our **AI recruiter** sources vetted specialists for any domain, on demand.” source ↗

Covers computer use“Full desktop and browser control, judged on the end state of long, multi-step tasks.” source ↗

Sells data“Expert-validated outputs that establish what successful completion looks like” source ↗

Covers computer use“Completing tasks across desktop applications, browsers, and tools” source ↗

Sells data“Email contact@refresh.dev to request environments or discuss a bespoke dataset” source ↗

Covers computer use“High-fidelity clones of full software suites, wiring together multiple connected applications, from EHRs to enterprise apps and beyond.” source ↗

Sells data“Snorkel helps frontier labs and AI teams develop specialized training data and environments that set their models and agents apart.” source ↗

Covers computer use“Evaluates computer-use agents on 108 long-horizon workflows.” source ↗

Sells data“We look for the most brilliant minds on the planet in every domain – doctors, lawyers, investment bankers, Fields Medalists, Harvard professors” source ↗

Covers computer use“teaching them to use computers, navigate the web, and reason for the first time.” source ↗

Covers computer use“CUB is the most diverse and representative benchmark for evaluating computer and browser use agents.” source ↗

Sells data“Our vast network of vetted coding experts offers scalable data production for the languages, coding domains, and programming expertise of your choice.” source ↗

Covers computer use“Isolated, containerized browsers and interactive web applications, instrumented for DOM/screen diffs and tool/API calls.” source ↗