MedQA A

AI benchmark · Knowledge

Built by
MIT Computer Science Artificial Intelligence Laboratory Jin et al
Domains
Knowledge
Tier
A why
Snapshot

Open benchmark ↗arxiv.org/e-print/2009.13081

What is MedQA?

MedQA measures how well models answer multiple-choice medical questions drawn from professional medical board exams in English, simplified Chinese and traditional Chinese.

Who built MedQA

Led by MIT Computer Science and Artificial Intelligence Laboratory (Jin et al.).

    Paper: arxiv.org/abs/2009.13081 ↗

    Lineage

    Benchmarks this one is built on and built from: versions, subsets, extensions and the indexes that combine them.

    Included in

    • Open Medical-LLM Leaderboard

      “A leaderboard that scores LLMs by accuracy on medical question-answering datasets including MedQA, MedMCQA, PubMedQA and medical MMLU subsets.”

    Tier A

    A

    Widely used: a lab reports it, or 50 or more models are scored on it.

    How much stands behind a tier: usage, who backs it, a confirmed line from its own site or paper, and a clear read on X. High with three of the four, medium with two, low otherwise.

    • 86 models scored
    • in curated RL directories (2 pts)

    On X · unclear, low buzz, from 1 post about MedQA in the last month

    MedQA-USMLE appears only as one score (94.21) cited in a model release announcement, with no discussion of the benchmark itself.

    @tylerrwayne ↗

    How we got to tier A
    1. Adoption: 86 models scored. Tier A.
    2. On X: unclear, low buzz. “MedQA-USMLE appears only as one score (94.21) cited in a model release announcement, with no discussion of the benchmark itself.” No move.
    3. In curated RL directories: 2 points, a floor of C.
    4. Confirmed: a line from its own site or paper places it.

    A clear view on X moves a tier one step up or down; most entries don't move. How tiers work.

    Where MedQA sits

    Its domain on the map, live. Hover a logo for its name; click to open it. Open it on the full map

    The galaxy could not load. See every vendor on the board.

    Next: browse all benchmarks. Entry from the RL Research daily scrape of public sources, 2026-11-07 snapshot.