PubMedQA B

AI benchmark · Biomedicine

Tier
B why
Snapshot

Open benchmark ↗pubmedqa.github.io/

What is PubMedQA?

PubMedQA measures a model's ability to answer biomedical research questions with yes/no/maybe using the corresponding PubMed abstracts, scored by accuracy and macro-F1 on a leaderboard.

Paper: arxiv.org/abs/1909.06146 ↗

Lineage

Benchmarks this one is built on and built from: versions, subsets, extensions and the indexes that combine them.

Included in

  • Open Medical-LLM Leaderboard

    “A leaderboard that scores LLMs by accuracy on medical question-answering datasets including MedQA, MedMCQA, PubMedQA and medical MMLU subsets.”

Tier B

B

Established: 15 or more models scored, or curated RL directories rate it.

How much stands behind a tier: usage, who backs it, a confirmed line from its own site or paper, and a clear read on X. High with three of the four, medium with two, low otherwise.

  • 23 models scored
  • in curated RL directories (1 pt)

On X · well regarded, low buzz, from 2 posts about PubMedQA in the last month

Mentioned as a widely cited medical QA benchmark by its creator in a job post and used by a small vendor to claim top results.

@DrQiaoJin ↗ · @qspro_ai ↗

How we got to tier B
  1. Adoption: 23 models scored. Tier B.
  2. On X: well regarded, low buzz. “Mentioned as a widely cited medical QA benchmark by its creator in a job post and used by a small vendor to claim top results.” No move.
  3. In curated RL directories: 1 point, a floor of none.
  4. Confirmed: a line from its own site or paper places it.

A clear view on X moves a tier one step up or down; most entries don't move. How tiers work.

Where PubMedQA sits

Its domain on the map, live. Hover a logo for its name; click to open it. Open it on the full map

The galaxy could not load. See every vendor on the board.

Next: browse all benchmarks. Entry from the RL Research daily scrape of public sources, 2026-11-14 snapshot.