MedQA A
AI benchmark · Knowledge
- Built by
- MIT Computer Science Artificial Intelligence Laboratory Jin et al
- Domains
- Knowledge
- Tier
- A why
- Snapshot
What is MedQA?
MedQA measures how well models answer multiple-choice medical questions drawn from professional medical board exams in English, simplified Chinese and traditional Chinese.
Health and bio mostly
“we present the first free-form multiple-choice OpenQA dataset for solving medical problems, MedQA, collected from the professional medical board exams.”
arxiv.org/abs/2009.13081 ↗
Who built MedQA
Led by MIT Computer Science and Artificial Intelligence Laboratory (Jin et al.).
Paper: arxiv.org/abs/2009.13081 ↗
Lineage
Benchmarks this one is built on and built from: versions, subsets, extensions and the indexes that combine them.
Included in
“A leaderboard that scores LLMs by accuracy on medical question-answering datasets including MedQA, MedMCQA, PubMedQA and medical MMLU subsets.”
Tier A
Widely used: a lab reports it, or 50 or more models are scored on it.
How much stands behind a tier: usage, who backs it, a confirmed line from its own site or paper, and a clear read on X. High with three of the four, medium with two, low otherwise.
- 86 models scored
- in curated RL directories (2 pts)
On X · unclear, low buzz, from 1 post about MedQA in the last month
MedQA-USMLE appears only as one score (94.21) cited in a model release announcement, with no discussion of the benchmark itself.
How we got to tier A
- Adoption: 86 models scored. Tier A.
- On X: unclear, low buzz. “MedQA-USMLE appears only as one score (94.21) cited in a model release announcement, with no discussion of the benchmark itself.” No move.
- In curated RL directories: 2 points, a floor of C.
- Confirmed: a line from its own site or paper places it.
A clear view on X moves a tier one step up or down; most entries don't move. How tiers work.
Where MedQA sits
Its domain on the map, live. Hover a logo for its name; click to open it. Open it on the full map
Next: browse all benchmarks. Entry from the RL Research daily scrape of public sources, 2026-11-07 snapshot.