Domain on the map · as Health and bio
Medical LLM benchmarks and RL environments for healthcare
Clinical and health-data tasks, such as working with patient records in standard formats and answering medical questions.
- RL environments
- 2
- AI benchmarks
- 148
- Shares entries with
- 22 categories
Snapshot 2026-11-05. How categories are built · Where the data comes from
In short
- What are RL environments for healthcare?
- A simulated workplace where an AI agent practises a task (a code repository, a browser, business software) and an automatic grader scores each attempt. Labs train on thousands of these attempts. Bought to improve models. For healthcare, that means clinical and health-data tasks, such as working with patient records in standard formats and answering medical questions.
- Who builds RL environments for healthcare?
- Surge AI and Anthromind.
- What are the main medical LLM benchmarks?
- Medmarks, ReXrank, BRIDGE Medical Leaderboard, Histopath-C, LABBench2 Clinical Trials and MEDIC Benchmark, among 142 confirmed below.
RL environments 2
- Surge AI
-
Anthromind
No line from its own site or paper places it in Healthcare yet; the catalog lists it here.
AI benchmarks 148
-
Medmarks
Tests health and bio“OPEN MEDICAL AI BENCHMARKS / BY MEDARC × SOPHONT” source ↗
-
ReXrank
Tests health and bio“ReXrank is a public leaderboard for chest X-ray image interpretation, including both radiology report generation (RRG) and visual question answering (VQA) tasks.” source ↗
-
BRIDGE Medical Leaderboard
Tests health and bio“we present BRIDGE, a comprehensive multilingual benchmark comprising 87 tasks sourced from real-world clinical data sources across nine languages.” source ↗
-
Histopath-C
Tests health and bio“we introduce Histopath-C, a new benchmark with realistic synthetic corruptions designed to mimic real-world distribution shifts observed in digital histopathology.” source ↗
-
LABBench2 Clinical Trials
Tests health and bio“LABBench2: An Improved Benchmark for AI Systems Performing Biology Research” source ↗
-
MEDIC Benchmark
Tests health and bio“we introduce MEDIC, a comprehensive evaluation framework establishing leading indicators of clinical LLM competence across five dimensions.” source ↗
-
Open Medical-LLM Leaderboard
Tests health and bio“providing a standardized platform for evaluating and comparing the performance of various large language models on a diverse range of medical tasks and datasets.” source ↗
-
CLEAR
Tests health and bio“We introduce the CLinical Evaluation of Ambiguity and Reliability (CLEAR) framework, which assesses how decision-space presentation, ambiguity, and uncertainty affect LLMs' reasoning on medical benchmarks.” source ↗
-
ASD-Bench
Tests health and bio“We introduce ASD-Bench, a systematic tabular benchmark evaluating ML, deep learning, and foundation model configurations across three age cohorts” source ↗
-
CardioLens
Tests health and bio“We introduce CardioLens, a leakage-resistant evaluation testbed for multi-sequence Cardiovascular Magnetic Resonance (CMR), constructed from private hospital archives” source ↗
-
CHI-Bench
No line from its own site or paper places it in Healthcare yet; the catalog lists it here.
-
DentalGPT Dental Eval Suite
Tests health and bio“To evaluate the capability of multimodal large language models (MLLMs) in understanding dental images, we curated a specialized evaluation dataset sourced from hospital and internet data.” source ↗
-
DrVD-Bench
Tests health and bio“we propose DrVD-Bench, the first multimodal benchmark for clinical visual reasoning.” source ↗
-
EHRBench
Tests health and bio“we introduce EHRBench, an automated and reliable EHR-grounded benchmark for evaluating LLM-based clinical decision-making at scale.” source ↗
-
FAM-Bench
Tests health and bio“We introduce FAM-Bench, a multi-modal Food-as-Medicine benchmark with 2500 nutrition-expert-verified instances across 13 diet-related health conditions.” source ↗
-
InvisibleBench
Tests health and bio“AI safety benchmark for long-term caregiving relationships. Tests crisis detection, regulatory compliance, and care quality across multi-turn conversations.” source ↗
-
VariantBench
Tests health and bio“We present VariantBench, a verifiable benchmark of 118 evaluations that tests agents on their ability to conduct variant discovery and interpretation.” source ↗
-
AgentRx
Tests health and bio“this work offers a new benchmark to support future developments relating to agentic systems in healthcare.” source ↗
-
BanglaMedVQA
Tests health and bio“we introduce BanglaMedVQA, a dataset comprising clinically validated image-question-answer pairs, along with a comprehensive evaluation of current foundation models on this resource.” source ↗
-
BLURB
Tests health and bio“BLURB comprises of a comprehensive benchmark for PubMed-based biomedical NLP applications, as well as a leaderboard for tracking progress by the community.” source ↗
-
CareQA-Vision
Tests health and bio“we introduce CareQA-Vision, a carefully curated vision benchmark derived from MIR and EIR exams, the residency entrance exams for medical and nursing specialists in Spain” source ↗
-
ClinicalMC
Tests health and bio“we propose ClinicalMC, a benchmark for multi-course clinical decision-making.” source ↗
-
CURE-Med
Tests health and bio“introducing CUREMED-BENCH, a high-quality multilingual medical reasoning dataset with open-ended reasoning queries with a single verifiable answer” source ↗
-
DeepTumorVQA
Tests health and bio“We propose DeepTumorVQA, a hierarchical benchmark that follows the multi-stage evidence chain in tumor diagnosis and decomposes 3D CT reasoning into four stages” source ↗
-
DermoBench
Tests health and bio“we establish DermoBench, a rigorous benchmark evaluating 11 tasks across four clinical axes: Morphology, Diagnosis, Reasoning, and Fairness” source ↗
-
DIYHealth Suite
Tests health and bio“we establish DIYHealthBench, the first benchmark to evaluate foundation models on home care tasks.” source ↗
-
EpiQAL
Tests health and bio“We present EpiQAL, the first diagnostic benchmark for epidemiological question answering across diverse diseases, comprising three subsets built from open-access literature.” source ↗
-
HealthBench-BR and PCDT-QA
Tests health and bio“We introduce HealthBench-BR, with 1,780 balanced true/false clinical assertions, and PCDT-QA, with 890 open-ended clinical questions scored by an LLM judge.” source ↗
-
IHBench
No line from its own site or paper places it in Healthcare yet; the catalog lists it here.
-
JMed48k
Tests health and bio“We introduce JMed48k, a multi-profession Japanese healthcare licensing benchmark for evaluating vision-language models.” source ↗
-
LingLanMiDian
Tests health and bio“LingLan is an expert-curated benchmark for evaluating large language models across five domains of Traditional Chinese Medicine” source ↗
-
LiveMedBench
Tests health and bio“A Live Medical Benchmark for Evaluating Large Language Models” source ↗
-
Med-RewardBench
Tests health and bio“we introduce Med-RewardBench, the first benchmark specifically designed to evaluate MRMs and judges in medical scenarios.” source ↗
-
MedConsultBench
Tests health and bio“a comprehensive framework designed to evaluate the complete online consultation cycle by covering the entire clinical workflow from history taking and diagnosis to treatment planning” source ↗
-
MedCTA
Tests health and bio“We introduce MedCTA, a benchmark for evaluating medical tool agents on clinician-validated, step-implicit tasks grounded in realistic multimodal clinical inputs” source ↗
-
MedGUIDE
Tests health and bio“we introduce MedGUIDE, a new benchmark for evaluating LLMs on their ability to make guideline-consistent clinical decisions.” source ↗
-
MedLayXPlain
Tests health and bio“we introduce MedLayXPlain, the first large-scale multimodal benchmark and evaluation framework for Medical Lay Language Generation (MLLG).” source ↗
-
MHGraphBench
Tests health and bio“we present a knowledge-graph (KG)-grounded benchmark for assessing LLMs on mental-health entity recognition, relation judgment, and two-hop reasoning.” source ↗
-
NeuroAtlas
Tests health and bio“42 datasets and 260k hours covering clinical EEG (epilepsy, sleep medicine, brain age estimation) and brain-computer interfaces” source ↗
-
PubMedQA
Tests health and bio“We introduce PubMedQA, a novel biomedical question answering (QA) dataset collected from PubMed abstracts.” source ↗
-
RxEval
Tests health and bio“We propose RxEval, a prescription-level benchmark that evaluates LLM prescribing capability by multiple-choice questions” source ↗
-
SycoEval-EM
Tests health and bio“a multi-agent simulation framework to evaluate LLM robustness to adversarial patient persuasion in emergency medicine” source ↗
-
MMLU Medical Genetics
Tests health and bio“The pattern of inheritance shown by malignant hyperthermia (hyperpyrexia) is:” source ↗
-
AcuityBench
Tests health and bio“We introduce AcuityBench, a benchmark for evaluating whether language models identify the appropriate urgency of care from user medical presentations.” source ↗
-
AgentClinic
Tests health and bio“we introduce AgentClinic, a multimodal agent benchmark for evaluating LLMs in simulated clinical environments” source ↗
-
AutoMedBench
Tests health and bio“Towards _Medical AutoResearch_ a benchmark for AI agents on medical research tasks.” source ↗
-
BCoughBench
Tests health and bio“No FM meets the clinical sensitivity threshold (Se@Sp95 ≥ 0.20) on most disease tasks under any BC sensor.” source ↗
-
CaseReportBench
Tests health and bio“We introduce CaseReportBench, an expert-annotated dataset for dense information extraction of case reports, focusing on IEMs.” source ↗
-
Checkup2Action
Tests health and bio“Checkup2Action provides a new multimodal benchmark for evaluating patient-oriented reasoning over clinical check-up reports.” source ↗
-
CheXpercept
Tests health and bio“CheXpercept contains 10,400 QA items derived from 2,100 CXRs, covering seven clinically critical pulmonary and cardiac lesions.” source ↗
-
ChronoMedKG
Tests health and bio“ChronoTQA, a benchmark of 3,341 questions across eight task types (six temporal plus two static controls)” source ↗
-
ClinConsensus
Tests health and bio“We introduce \textsc{ClinConsensus}, a Chinese medical benchmark of 2{,}500 expert-curated cases spanning 36 specialties” source ↗
-
EHR-Complex
Tests health and bio“we introduce EHR-Complex, a large-scale benchmark designed for interactive clinical database reasoning.” source ↗
-
Enhancing Pathological VLMs with Cross-scale Reasoning
Tests health and bio“we construct Scale-VQA, a high-quality benchmark with 4,685 multiple-choice questions grounded in 2,537 pathology images across multiple magnification levels.” source ↗
-
EpiBench
Tests health and bio“EpiBench defines five clinically motivated tasks spanning clinical decision-making, EEG report generation, pharmacogenomic precision medicine, treatment recommendation, and deep research planning.” source ↗
-
EQUITRIAGE
Tests health and bio“We present EQUITRIAGE, a fairness audit of LLM-based ESI assignment evaluating five models” source ↗
-
HiMed: Incentivizing Hindi Reasoning in Medical LLMs
Tests health and bio“we introduce HiMed, a Hindi reasoning medical corpus and benchmark suite covering both Western and Indian medicine.” source ↗
-
MedAgentBench
Tests health and bio“MedAgentBench is a comprehensive evaluation suite designed to benchmark the agent capabilities of large language models (LLMs) in medical records settings.” source ↗
-
MedHELM
Tests health and bio“The benchmarks represent a spectrum of healthcare scenarios, from diagnostic decision-making to patient communication” source ↗
-
MultiMedQA
Tests health and bio“we present MultiMedQA, a benchmark combining six existing open question answering datasets spanning professional medical exams, research, and consumer queries” source ↗
-
Omi SOAP Note Safety Benchmark
Tests health and bio“300 synthetic doctor-patient dialogues with sentence-level IDs (SIDs) for evidence tracking. Each model generates a SOAP note from the same transcript.” source ↗
-
PhysicianBench
Tests health and bio“We introduce PhysicianBench, a benchmark for evaluating LLM agents on physician tasks grounded in real clinical setting within electronic health record (EHR) environments.” source ↗
-
R2MED
Tests health and bio“we introduce R2MED, the first benchmark explicitly designed for reasoning-driven medical retrieval.” source ↗
-
RXL-RADSet
Tests health and bio“To create RXL-RADSet, a radiologist-verified synthetic multi-RADS benchmark, and compare validity and accuracy of open-weight small language models” source ↗
-
ABRA: Agent Benchmark for Radiology Applications
Tests health and bio“A benchmark for evaluating (V)LLM agents completing radiology workflows inside a medical imaging viewer.” source ↗
-
AgentEHR
Tests health and bio“AGENTEHR challenges agents to perform complex clinical decision-making—such as diagnosis and treatment planning—directly within raw, high-noise EHR databases.” source ↗
-
AIPatient Arena
Tests health and bio“we propose AIPatient Arena, an EHRs-grounded evaluation framework for assessing the clinical utility of LLMs across eight dimensions of clinical competence.” source ↗
-
Causal Sensitivity Score
Tests health and bio“a pre-registered interventional metric that mutates oncology tumor-board cases along five clinically meaningful dimensions” source ↗
-
CBT-Audio
Tests health and bio“We introduce CBT-Audio, a dataset for evaluating patient distress estimation from spoken CBT sessions with audio language models.” source ↗
-
Clinical Reasoning Graphs
Tests health and bio“We apply this pipeline to 750 traces from five LLMs across 50 New England Journal of Medicine Clinicopathological Conference cases” source ↗
-
ClinicalBench
Tests health and bio“ClinicalBench is a 400-question test over 43 MIMIC-IV patients across 9 assertion-sensitive categories.” source ↗
-
ClinPivot
Tests health and bio“We introduce ClinPivot, an auditable treatment-decision benchmark built from biomedical relations and pivoted patient contexts.” source ↗
-
CodeClinic
Tests health and bio“We introduce CodeClinic, a benchmark built on MIMIC-IV for evaluating whether LLM agents can synthesize and compose reusable clinical skills instead of relying on fixed toolboxes.” source ↗
-
CPCD-Bench
Tests health and bio“We further build CPCD-Bench to evaluate models' long-horizon campus counseling capabilities from three dimensions: session-level response, long-horizon memory recall, and temporal-causal reasoning.” source ↗
-
DrugAudit
Tests health and bio“We also contribute DrugAudit, a 3,772-item authority-aware benchmark” source ↗
-
GlobalDentBench
Tests health and bio“Here we introduce GlobalDentBench, the first multinational dental benchmark, featuring a taxonomy that encompasses 14 dental specialties across 88 countries and regions spanning six continents.” source ↗
-
HealthAdminBench
Tests health and bio“HealthAdminBench contains 135 tasks sourced from three core healthcare administrative workflows (prior authorization, DME orders, appeals) across three difficulty levels.” source ↗
-
ICU-Bench
Tests health and bio“ICU-Bench contains 1,000 privacy-sensitive profiles from two document domains, medical reports and labor contracts” source ↗
-
IMCBench
Tests health and bio“we introduce IMCBench, an image-grounded, multi-turn medical conversation benchmark that pairs real, publicly available clinical images with synthetic patient profiles” source ↗
-
LifeAgentBench
Tests health and bio“a large-scale QA benchmark for long-horizon, cross-dimensional, and multi-user lifestyle health reasoning, containing 22,573 questions” source ↗
-
LLMEval-Med
Tests health and bio“we present LLMEval-Med, a new benchmark covering five core medical areas, including 2,996 questions created from real-world electronic health records” source ↗
-
LungCURE
Tests health and bio“We introduce LungCURE, the first standardized multimodal benchmark built from 1,000 real-world, clinician-labeled cases across more than 10 hospitals.” source ↗
-
Med-HALT
Tests health and bio“We propose a new benchmark and dataset, Med-HALT (Medical Domain Hallucination Test), designed specifically to evaluate hallucinations.” source ↗
-
MedBench v5
Tests health and bio“We introduce MedBench v5, a redesigned benchmark for clinical multimodal models (language, vision-language, and agent systems)” source ↗
-
MedFrameQA
Tests health and bio“We present MedFrameQA, the first benchmark explicitly designed to test multi-image medical VQA through educationally-validated diagnostic sequences.” source ↗
-
Medical Chronology LLM Benchmark
Tests health and bio“benchmarking LLMs on medical chronology extraction — a critical task in medical-legal case review.” source ↗
-
Medical Information Response Audit
Tests health and bio“a bilingual, controlled benchmark that assesses whether LLMs provide comparable medical information across user-side language, register, and health literacy signals.” source ↗
-
MedMCQA
Tests health and bio“More than 194k high-quality AIIMS & NEET PG entrance exam MCQs covering 2.4k healthcare topics and 21 medical subjects are collected” source ↗
-
MedMemoryBench
Tests health and bio“MedMemoryBench: Benchmarking Agent Memory in Personalized Healthcare” source ↗
-
MedMeta
Tests health and bio“we introduce MedMeta, the first benchmark designed to evaluate an LLM's ability to generate conclusions from medical meta-analyses using only the abstracts of cited studies.” source ↗
-
MedMosaic
Tests health and bio“we present MedMosaic, a medical audio question-answering dataset designed to benchmark language and audio reasoning models under realistic clinical constraints.” source ↗
-
MedRedFlag
Tests health and bio“we investigate how LLMs react to false premises embedded within real-world health questions.” source ↗
-
MedRepBench
Tests health and bio“We introduce MedRepBench, a comprehensive benchmark built from 1,900 de-identified real-world Chinese medical reports spanning diverse departments, patient demographics, and acquisition formats.” source ↗
-
MedSafe-Dx
Tests health and bio“given a GP intake summary for a synthetic patient, does a language model flag the patient who needs prompt attention and leave alone the one who does not?” source ↗
-
MedStruct-S
Tests health and bio“MedStruct-S contains 3,582 annotated real-world clinical report pages.” source ↗
-
MLB
Tests health and bio“we introduce a Medical LLM Benchmark MLB, a comprehensive benchmark evaluating LLMs on both foundational knowledge and scenario-based reasoning.” source ↗
-
MMed-Bench-IR
Tests health and bio“We introduce MMed-Bench-IR, a benchmark designed to disentangle these axes across 6 languages and three structurally heterogeneous tasks” source ↗
-
MRI-Eval
Tests health and bio“a tiered benchmark for relative model comparison on MRI physics and GE scanner operations knowledge using primary multiple-choice questions (MCQ)” source ↗
-
MTBBench
Tests health and bio“We introduce MTBBench, an agentic benchmark simulating MTB-style decision-making through clinically challenging, multimodal, and longitudinal oncology questions.” source ↗
-
Oncology Report-Derived VQA
Tests health and bio“generates multiple-choice VQA datasets directly from paired private radiology reports and 3D oncology imaging” source ↗
-
PlaceboBench
Tests health and bio“PlaceboBench is a hallucination benchmark for retrieval-augmented generation (RAG) in the pharmaceutical domain.” source ↗
-
PMC-VQA
Tests health and bio“we propose a test set that has undergone manual verification, which is significantly more challenging, serving to better monitor the development of generative MedVQA methods.” source ↗
-
Pocket-Dentist
Tests health and bio“Here we present Pocket-Dentist, an efficiency-aware benchmark for dental multimodal question answering” source ↗
-
PsychEthicsBench
Tests health and bio“the first principle-grounded benchmark based on Australian psychology and psychiatry guidelines, designed to evaluate LLMs' ethical knowledge and behavioral responses” source ↗
-
UCSF-PDGM-VQA
Tests health and bio“We introduce a clinically relevant visual question answering (VQA) benchmark -- the UCSF-PDGM-VQA dataset -- consisting of 2,387 QA pairs from 473 glioma-related MRI studies” source ↗
-
VERA-MH
Tests health and bio“a novel clinically-validated evaluation for safety of chatbots in the context of mental health support” source ↗
-
VietMed-MCQ
Tests health and bio“we introduce VietMed-MCQ, a novel multiple-choice question dataset generated via a Retrieval-Augmented Generation (RAG) pipeline” source ↗
-
When Cases Get Rare
Tests health and bio“we introduce OGCaReBench, a free-form retrieval-focused benchmark aimed at evaluating LLMs at answering clinical questions that require going beyond typical guidelines.” source ↗
-
When Multiple Scripts Matter
Tests health and bio“we introduce MultiClin, a clinical ASR benchmark designed to evaluate robustness to multiscript variability.” source ↗
-
MedHal-Loc
Tests health and bio“A **localization-faithfulness benchmark and metric** for medical hallucination detectors” source ↗
-
RSPC
Tests health and bio“1,799 Reddit posts annotated by psychiatrists for diagnostic categories, including the most prevalent mood disorders (anxiety and depression)” source ↗
-
FHIR-AgentBench
Tests health and bio“we introduce FHIR-AgentBench, a benchmark that grounds 2,931 real-world clinical questions in the HL7 FHIR standard.” source ↗
-
Are LLMs Ready to Assist Physicians? PhysAssistBench for Interactive Doctor-Patient-EHR Assistance
Tests health and bio“We introduce PhysAssistBench, a benchmark for interactive doctor-patient-EHR assistance. Built from real MIMIC-IV cases” source ↗
-
ART (Action-based Reasoning Task)
Tests health and bio“We introduce ART, an Action-based Reasoning clinical Task benchmark for medical AI agents, which mines real-world EHR data to create challenging tasks” source ↗
-
BioASQ
Tests health and bio“BioASQ organizes challenges on biomedical semantic indexing and question answering (QA).” source ↗
-
DALPHIN
Tests health and bio“We created DALPHIN, the first multicentric open benchmark for pathology AI copilots, comprising 1236 images from 300 cases, spanning 130 rare to common diagnoses” source ↗
-
Efficient Multimodal Clinical Question Answering for Pulmonary Embolism Risk Assessment
Tests health and bio“We formulate eight diagnostic and prognostic tasks as structured clinical question answering problems and evaluate on typical efficient MLLMs” source ↗
-
Health Memory Arena
Tests health and bio“An open, live leaderboard for healthcare AI agents, supporting continuous evaluation and rolling updates” source ↗
-
MedVidBench
Tests health and bio“we first introduce \textbf{MedVidBench}, a large-scale benchmark of 531,850 video-instruction pairs across 8 medical sources spanning video, segment, and frame-level tasks” source ↗
-
PulseMind
Tests health and bio“we develop the PulseMind Benchmark, a multi-turn diagnostic consultation benchmark with a four-dimensional evaluation protocol” source ↗
-
Active Evidence-Seeking and Diagnostic Reasoning in Large Language Models for Clinical Decision Support
Tests health and bio“we introduce an OSCE-inspired standardized patient simulator and a controlled, reproducible benchmark for active diagnostic inquiry.” source ↗
-
CounselBench
Tests health and bio“We present CounselBench, a large-scale benchmark developed with 100 mental health professionals to evaluate and stress-test large language models (LLMs) in realistic help-seeking scenarios.” source ↗
-
EHRStruct
Tests health and bio“we introduce EHRStruct, a benchmark specifically designed to evaluate LLMs on structured EHR tasks.” source ↗
-
Evaluating the Pre-Consultation Ability of LLMs using Diagnostic Guidelines
Tests health and bio“we aim to contribute to the evaluation and further development of LLM applications in real-world clinical settings.” source ↗
-
Generating synthetic computed tomography for radiotherapy
Tests health and bio“SynthRAD2025 benchmarked sCT methods on 2,362 patients from five European centers across head and neck, thorax, and abdomen.” source ↗
-
HAO / TBFact
Tests health and bio“we propose TBFact, a ``model-as-a-judge'' framework designed to assess the comprehensiveness and succinctness of generated summaries.” source ↗
-
Large Language Models Lack Temporal Awareness of Medical Knowledge
Tests health and bio“we built TempoMed-Bench, the first-of-its-kind benchmark for evaluating the temporal awareness of the LLMs in the medical domain through evolving guideline knowledge.” source ↗
-
MedBrowseComp
Tests health and bio“the first benchmark that systematically tests an agent’s ability to reliably retrieve and synthesize multi-hop medical facts from live, domain-specific knowledge bases” source ↗
-
MedCase-Structured
Tests health and bio“Evaluation on MedCase-Structured reveals consistently lower diagnostic accuracy for LLMs on structured FHIR inputs than with plain text” source ↗
-
MultiTurnPSB
Tests health and bio“MultiTurnPSB: Evaluating Multi-Turn Jailbreak Attacks an dClassifier-Based Defenses for Medical AI Safety” source ↗
-
RealICU
Tests health and bio“We introduce RealICU, a hindsight-annotated benchmark for evaluating large language models (LLMs) under realistic ICU conditions” source ↗
-
RubricsTree
Tests health and bio“Scalable and Evolving Open-Ended Evaluation of Personal Health Agents across Health Memory and Medical Skills” source ↗
-
SlakeVQA
Tests health and bio“we present a large bilingual dataset, SLAKE, with comprehensive semantic labels annotated by experienced physicians and a new structural medical knowledge base for Med-VQA.” source ↗
-
SuiChat-CN
Tests health and bio“We introduce SuiChat-CN, a Chinese group-chat benchmark for contextual suicide risk assessment.” source ↗
-
SurgGoal
Tests health and bio“we introduce a multicentric meta-evaluation benchmark with valid procedural variations and invalid plans containing order and content errors.” source ↗
-
Synthesis and Evaluation of Long-term History-aware Medical Dialogue
Tests health and bio“We establish three benchmark tasks-In-dialogue Reasoning, Cross-dialogue Reasoning, and Synthesis Reasoning-to evaluate the memory capabilities of healthcare agents.” source ↗
-
Veterinary Clinical Summarization Eval
Tests health and bio“We evaluated three commercially available veterinary-focused LLM summarization tools (Product 1 [Hachiko] and Products 2 and 3) on a standardized dataset of veterinary oncology records.” source ↗
-
WBCMor VQA
Tests health and bio“a clinically validated bilingual English, Urdu morphology aware VQA benchmark for leukemia and normal white blood cell analysis.” source ↗
-
What Does the AI Doctor Value? Auditing Pluralism in the Clinical Ethics of Language Models
Tests health and bio“We present a framework for auditing value pluralism in medical AI, comprising a benchmark of clinician-verified dilemmas” source ↗
-
When Audio-Language Models Fail to Leverage Multimodal Context for Dysarthric Speech Recognition
Tests health and bio“Recent audio-language models raise the possibility of improving performance by conditioning on additional clinical context at inference time” source ↗
-
When Symptoms Are Not Enough
Tests health and bio“We introduce a SCID-anchored benchmark of 555 semi-structured experiential interviews paired with diagnostic reference labels for anxiety disorder, major depressive disorder” source ↗
-
Towards Error-Free EHRs
Tests health and bio“we introduce EHR-ReasonCon, a reasoning-intensive benchmark for note-table consistency verification. Built on MIMIC-III with expert-guided annotations” source ↗
6 not yet confirmedListed in the catalog; no line from its own site or paper confirms it yet.
Data 13
- Pareto
- AfterQuery
- Align AI
- Aptura
- Mercor
- ReasonCore
- Refresh
- Scale AI
- Surge AI
- Symbal
-
Theta
Covers health and bio“Analyze Patient Labs” source ↗
- Turing
-
Vetto AI
Sells data“Data grounded in real expertise, from practitioners at the frontier of the sciences, medicine, law, finance, engineering, and the humanities.” source ↗
Covers health and bio“from practitioners at the frontier of the sciences, medicine, law, finance, engineering, and the humanities.” source ↗
Next: pick another domain. Tags come from the RL Research daily scrape, canonicalized nightly.