IOI A

Name: IOI
Creator: Vals AI
Keywords: Coding

AI benchmark · Coding

Publisher: Vals AI
Domains: Coding
On X: @ValsAI ↗
Tier: A why
Snapshot: October 11, 2026

Latest from Vals AI on X

@ValsAI ↗

Do agent teams pay off? We ran GPT-6 Sol and Claude Opus 5.5 on Vibe Code Bench, alone and as teams, at medium and max reasoning effort. Teams cost 1.8× to 5.1× more, but only one of the four team-vs-single comparisons produced a statistically significant improvement: Sol at medium effort, which gained 7.3 points.
— ValsAI (@ValsAI) Oct 9, 2026
What happens when a teenager pushes back on an AI’s safety guardrails? 1 in 5 teens now talk to AI chatbots about mental health problems, but these chatbots can make mistakes. Some of the most dangerous mistakes happen later in conversations, after the model initially responds appropriately. We built the SAFE-Teen benchmark to test how models handle serious conversations with adolescents.
— ValsAI (@ValsAI) Oct 8, 2026
We tested Mercury Decide, a new model from @_inception_ai, on the same benchmark we used for TypeSafe's Jev. It matched frontier accuracy on claim verification at the lowest cost we have measured so far.
— ValsAI (@ValsAI) Oct 8, 2026

What is IOI?

Based on the International Olympiad in Informatics

Tier A

Widely used: a lab reports it, or 50 or more models are scored on it.

How much stands behind a tier: usage, who backs it, a confirmed line from its own site or paper, and a clear read on X. High with three of the four, medium with two, low otherwise.

57 models scored
in curated RL directories (2 pts)

On X · mixed, moderate buzz, from 10 posts about IOI in the last month

Labs report gold-level and perfect IOI scores (Gemini 4, dots3, Nemotron), but perfect 2024-2026 results suggest the benchmark is nearing saturation at the top.

@007GameIOI ↗ · @ValsAI ↗ · @ChaoQiao42 ↗

How we got to tier A

Adoption: 57 models scored. Tier A.
On X: mixed, moderate buzz. “Labs report gold-level and perfect IOI scores (Gemini 4, dots3, Nemotron), but perfect 2024-2026 results suggest the benchmark is nearing saturation at the top.” No move.
In curated RL directories: 2 points, a floor of C.
Not yet confirmed by a line from its own site or paper.

A clear view on X moves a tier one step up or down; most entries don't move. How tiers work.

Not on the map yet: no line from its own site or paper places it in a domain.

More Coding AI benchmarks

Coding RL environments

Next: browse all benchmarks. Entry from the RL Research daily scrape of public sources, 2026-10-11 snapshot.