unix-ctf B
AI benchmark · ML
Latest from Vmax on X
@VmaxAI ↗What is unix-ctf?
unix-ctf procedurally generates capture-the-flag tasks in Linux containers to train and evaluate shell agents' Unix competence, scored by solve rate on recovering hidden flags.
Security mostly
“unix-ctf is a procedural generator of capture-the-flag tasks for shell agents.”
arxiv.org/abs/2605.29115 ↗“We introduce unix-ctf, a procedural generator of capture-the-flag environments for training and evaluating Unix competence in language-model shell agents.”
vmax.ai/research ↗
Paper: arxiv.org/abs/2605.29115 ↗
Tier B
Established: 15 or more models scored, or curated RL directories rate it.
How much stands behind a tier: usage, who backs it, a confirmed line from its own site or paper, and a clear read on X. High with three of the four, medium with two, low otherwise.
- built by Vmax
- in curated RL directories (5 pts)
How we got to tier B
- Adoption: built by Vmax. Tier C.
- Not yet read on X.
- In curated RL directories: 5 points, a floor of B. Raised to B.
- Confirmed: a line from its own site or paper places it.
A clear view on X moves a tier one step up or down; most entries don't move. How tiers work.
Where unix-ctf sits
Its realm in the universe, live. Hover a star for its name; click to open it. Open it in the universe
Next: browse all benchmarks. Entry from the RL Research daily scrape of public sources, 2026-10-11 snapshot.