CWE-bench v1 C

Name: CWE-bench v1
Creator: Collinear
Keywords: Coding,Computer Use,Enterprise Workflows,Long-Horizon

AI benchmark · Coding

Publisher: Collinear
Domains: Coding Computer Use Enterprise Workflows Long-Horizon
On X: @collinearAI ↗
Tier: C why
Snapshot: October 11, 2026

Open benchmark ↗collinear.ai/research

Full listing name: CWE-bench v1 (2026-09, cwe-bench.com; part of the Artificial Analysis Cyber Index)

Latest from Collinear on X

@collinearAI ↗

Introducing MatFlowBench, our benchmark for LLM agents running materials science workflows. Agents carry out first-principles calculations or design budget-limited measurement campaigns, then draw conclusions the resulting data support under the task’s stated physical assumptions. Website: https://t.co/BFeISrsF1p Blogpost:
— collinearAI (@collinearAI) Oct 9, 2026
Heading to #COLM2026? Let's meet! We’re bringing the team behind YC-Bench, CWE-bench, and FrontierCUA. Meet us at our YC-Bench poster and at booth 304, Oct. 6–9 in San Francisco. Link in the first comment.
— collinearAI (@collinearAI) Oct 2, 2026
Gemini 4 Argon ties for #1 on CWE-bench v1. 68% programmatic pass@1 on 120 held-out audit-and-patch tasks, level with Grok 4.7 and GPT-6 Astra.
— collinearAI (@collinearAI) Oct 2, 2026

What is CWE-bench v1?

CWE-bench measures coding agents' ability to handle real vulnerabilities in real codebases across 73 CWEs and all 10 OWASP categories, with programmatic and judge verification.

Security mostly
“CWE-bench is a frontier benchmark for AI Cybersecurity”
collinear.ai/research ↗
Coding
“Coding agents against real vulnerabilities in real codebases, with strict gates and both programmatic and judge verification on every task.”
collinear.ai/research ↗

Who built CWE-bench v1

Collinear built it
“Evaluations we build to find where models fail”
collinear.ai/research ↗

Tier C

Niche: a handful of models scored, or built by an RL vendor.

How much stands behind a tier: usage, who backs it, a confirmed line from its own site or paper, and a clear read on X. High with three of the four, medium with two, low otherwise.

built by Collinear
in curated RL directories (2 pts)

How we got to tier C

Adoption: built by Collinear. Tier C.
Not yet read on X.
In curated RL directories: 2 points, a floor of C.
Confirmed: a line from its own site or paper places it.

A clear view on X moves a tier one step up or down; most entries don't move. How tiers work.

Where CWE-bench v1 sits

Its realm in the universe, live. Hover a star for its name; click to open it. Open it in the universe

The galaxy could not load. See every vendor on the board.

More Coding AI benchmarks

Coding RL environments

Next: browse all benchmarks. Entry from the RL Research daily scrape of public sources, 2026-10-11 snapshot.