VADER C

Name: VADER
Creator: AfterQuery
Keywords: Coding,Computer Use,Enterprise Workflows

AI benchmark · Coding

Publisher: AfterQuery
Domains: Coding Computer Use Enterprise Workflows
On X: @afterquery ↗
Tier: C why
Snapshot: October 11, 2026

Open benchmark ↗afterquery.com/research

Latest from AfterQuery on X

@afterquery ↗

Not all coding data is created equal. Using only 500 tasks from @AfterQuery's SWE agent dataset, our internal post-training team was able to improve Qwen3.8-27B-Medium by 11.3 points in only 15 steps of GRPO. Read more about our training experiment in the blog post linked in the comments.
— afterquery (@afterquery) Oct 8, 2026
Congratulations to the @motif_tech team on the release of Motif 3! AfterQuery is proud to have served as the sole data partner on this model.
— afterquery (@afterquery) Aug 24, 2026
Congrats to the @WeAreLegora team on the release of the Legora BAR, a benchmark for agentic legal work built on 5,161 real law firm cases across 28 practice areas! @AfterQuery helped QA the benchmark with Legora. Proud to work with their team on measuring the frontier of agentic legal work. Link to the full blog post in the replies.
— afterquery (@afterquery) Jul 28, 2026

What is VADER?

VADER measures how well LLMs assess, detect, explain, and remediate real-world software vulnerabilities, scored by human security experts.

Security mostly
“a human-evaluated benchmark designed explicitly to assess LLM performance across four key vulnerability-handling dimensions: assessment, detection, explanation, and remediation.”
arxiv.org/abs/2505.19395 ↗
Coding
“VADER comprises 174 real-world software vulnerabilities, each carefully curated from GitHub repositories and annotated by security experts.”
arxiv.org/abs/2505.19395 ↗

Paper: arxiv.org/abs/2505.19395 ↗

Tier C

Niche: a handful of models scored, or built by an RL vendor.

How much stands behind a tier: usage, who backs it, a confirmed line from its own site or paper, and a clear read on X. High with three of the four, medium with two, low otherwise.

built by AfterQuery
in curated RL directories (2 pts)

How we got to tier C

Adoption: built by AfterQuery. Tier C.
Not yet read on X.
In curated RL directories: 2 points, a floor of C.
Confirmed: a line from its own site or paper places it.

A clear view on X moves a tier one step up or down; most entries don't move. How tiers work.

Where VADER sits

Its realm in the universe, live. Hover a star for its name; click to open it. Open it in the universe

The galaxy could not load. See every vendor on the board.

More Coding AI benchmarks

Coding RL environments

Next: browse all benchmarks. Entry from the RL Research daily scrape of public sources, 2026-10-11 snapshot.