MMLU C

Lineage

Benchmarks this one is built on and built from: versions, subsets, extensions and the indexes that combine them.

Derived benchmarks

• Edge LLM Leaderboard: Raspberry Pi 5 extension

  “Benchmarks throughput, memory/model size, and MMLU accuracy of quantized LLMs running locally on a Raspberry Pi 5 (8GB) edge device.”

• HELM MMLU extension

  “HELM's MMLU leaderboard scores language models by exact-match accuracy on the Massive Multitask Language Understanding multiple-choice knowledge exam, within Stanford CRFM's holistic evaluation framework.”

• MMLU Medical Genetics subset

• MMLU Professional Medicine subset

• MMLU-Pro extension

• MMLU-ProX extension

• MMLU-Redux extension

• MMLU-STEM subset

• MMMLU extension

  “MMMLU measures multilingual general knowledge by testing models on the MMLU test set professionally translated into 14 languages.”

• Open Italian LLM Leaderboard extension

  “A leaderboard that scores open LLMs on Italian-language benchmarks translated into Italian, such as MMLU, ARC Challenge and HellaSwag.”

• Open Multilingual LLM Evaluation Leaderboard extension

  “A leaderboard ranking LLMs across 29 mostly non-English languages on machine-translated ARC, HellaSwag, MMLU and TruthfulQA.”

Included in

• Open Medical-LLM Leaderboard

  “A leaderboard that scores LLMs by accuracy on medical question-answering datasets including MedQA, MedMCQA, PubMedQA and medical MMLU subsets.”