Directory label

Multimodal: AI benchmarks

A label other catalogs put on entries, not one of the domains on our map. On the map, each entry here sits in the domain its own site or paper leads with.

Tasks that combine text with images, video or other media, where a model has to ground its answer in what it sees.

AI benchmarks
254
Shares entries with
2 categories

Snapshot 2026-11-05. How categories are built · Where the data comes from

In short

What are the main multimodal benchmarks?
Chartography, SAGE, CharXiv, BabyVision, BenchCAD Vision2Code and MMMU-Pro, among 235 confirmed below.

Only benchmarks use the Multimodal tag: the RL environment directories file their vendors under other categories. Browse all RL environments.

AI benchmarks 254

  1. ChartographySurge AI
  2. SAGEVals AI
  3. CharXiv
  4. BabyVisionUniPat AI
  5. BenchCAD Vision2Code
  6. MMMU-ProXiang Yue, Wenhu Chen, Graham Neubig et al
  7. OmniDocBench 1.5OpenDataLab
  8. Video-MME
  9. MathVision
  10. ScreenSpot Pro
  11. ZeroBench
  12. MMSearch-Plus
  13. Visual-Language Understanding
  14. MortgageTaxVals AI
  15. MathVistaMicrosoft, University of California, University of Washington, Los Angeles
  16. Design ArenaArcada Labs
  17. Ego3D-Bench
  18. ScienceQA
  19. BC-VL
  20. BenchCAD Vision2Code-1000
  21. ChartMuseum
  22. ChartQAPro
  23. CharXiv-D
  24. CharXiv-R
  25. CountQA
  26. GDPval-MMMiniMax
  27. HiPhO
  28. MMBC
  29. MMSearchPlus
  30. SURDS
  31. SVFSearch
  32. TVBench
  33. UniGenBench
  34. UniGenBench English Long
  35. VideoMME w/o sub.
  36. VideoMME w sub.
  37. VisFactor
  38. VLADBench
  39. WorldVQA
  40. Video SimpleQA
  41. OfficeQA ProDatabricks
  42. DynaMath
  43. CC-OCR
  44. Blueprint-Bench 2Andon Labs
  45. AudioMC – Text Output
  46. DocVQACVC, IIIT Hyderabad, Universitat Autònoma de Barcelona
  47. OCRBench V2Huazhong University of Science, Technology
  48. OmniDocBenchOpenDataLab
  49. PerceptionBenchMoonshot AI
  50. RealWorldQAxAI
  51. MMLongBench-Doc
  52. ERQAGoogle DeepMind
  53. MLVU
  54. LVBench
  55. MMVU
  56. TIR-Bench
  57. MedXpertQA
  58. MedXpertQA MM
  59. Arena AI Image-to-VideoLMArena
  60. CADGenBenchHugging Face
  61. DataComp-VLMmlfoundations
  62. MMAUUniversity of Maryland, College Park
  63. MMSI-BenchShanghai AI Lab
  64. ProductWebGenSJTU DENG Lab
  65. Visual Aesthetic BenchmarkBakeLab
  66. VTB
  67. BLINK
  68. Facet-Probe
  69. FraudBench
  70. GenAI-BenchTIGER-AI-Lab
  71. Math-VRMeituan, HKU MMLab, CUHK
  72. MMBench-Video
  73. MMEB
  74. MMGist
  75. MuirBench
  76. olmOCR-bench
  77. Physical AI Bench Generation
  78. Physical AI Bench Understanding
  79. Video-HolmesARC Lab, Tencent PCG, City University of Hong Kong
  80. VRBenchOpenGVLab
  81. WikiVQABench
  82. ZEROBench-Sub
  83. OfficeQADatabricks
  84. CountBench
  85. Design2Code
  86. ODINW13
  87. VideoMMMU
  88. Vision2WebZhipu AI
  89. SimpleVQA
  90. AudioMC – Audio OutputScale AI
  91. ChartQA
  92. InfoVQA
  93. TextVQAFacebook AI Research
  94. AtelierEval
  95. AutoEval-Video
  96. HLV-1K
  97. MMStarShanghai AI Lab, USTC, CUHK
  98. PaintBenchNew York University
  99. SpeedrunBench: SuperTux 1-1Patronus AI
  100. TinyGrafixBenchPaintBench, Kai Xu, Saining Xie et al, Ellis Brown
  101. Video-MME v2
  102. VideoDR
  103. CADBench
  104. EmoMM
  105. GroundUI-1K
  106. IMAVB
  107. InfiniBenchVision-CAIR
  108. InfoVQAtest
  109. InkBench OCR
  110. JMMMU-Pro
  111. Judge Model for Large-scale Multimodality BenchmarksUniversity of Southern California
  112. LaViSA
  113. LingoQA
  114. LiveK12Bench
  115. MDPBench
  116. MM-MT-Bench
  117. MM-OptBench
  118. MMCL-Bench
  119. MMDG-BenchZhengzhou University, MBZUAI, Tampere University, ELLIS Institute Finland
  120. MomentSeeker
  121. MultiEmo-Bench
  122. MVRB
  123. NuscenenuTonomy
  124. OCR Bench Britannica
  125. OmniClean
  126. OmniRef-Bench
  127. PAL-Bench
  128. Physical AI Bench Conditional Generation
  129. RTV-Bench
  130. SVHallucKAIST
  131. TempCompass
  132. UniREditBenchUnifiedReward, Shanghai Innovation Institute
  133. VBVR-BenchVideo-Reason, Nanyang Technological University, multi-university collaboration incl. Carnegie Mellon University
  134. Vibe-Eval
  135. VideoFDBNVIDIA
  136. VisIT-Bench Single Image
  137. VPCT
  138. V-STaR
  139. VT-Bench
  140. WebMainBenchOpenDataLab
  141. WHOOPS Explanation of Violation
  142. WorldArena Track 2
  143. WorldScore
  144. ExtractBenchLlamaIndex
  145. ExtractBench ProLlamaIndex
  146. Flame-VLM-Code
  147. A-OKVQA
  148. OfficeQA Pro v2Databricks
  149. GDP.pdf w/ Python
  150. MMMU
  151. olmOCR
  152. VoxPopuli WER
  153. Design Arena WebsiteIntelligence Company
  154. MathVision w/ Python
  155. WorldVQA ForceAnswer
  156. We-Math
  157. RefCOCO
  158. ImageMiningZ.ai
  159. MMSearch
  160. Facts-VLM
  161. CharXiv w/o tools
  162. OmniDocBench 1.6OpenDataLab
  163. AudioMC
  164. Humanity's Sixth SenseScale AI, Elorian
  165. ViDoRe V3 Image
  166. ViDoRe V3 Markdown
  167. Vals Multimodal IndexVals AI
  168. VoxelBench Text
  169. VoxelBench Image
  170. MathVista test
  171. CharXiv Extended
  172. MedXpertQA MM Extended
  173. MMMU-Pro Extended
  174. Arena AI Image Edit
  175. Arena AI Text-to-Image
  176. Arena AI Text-to-VideoLMArena
  177. Arena AI Video Edit
  178. Artificial Analysis Image Editing Arena
  179. Artificial Analysis Image-to-Video Arena
  180. Artificial Analysis Text-to-Image Arena
  181. Artificial Analysis Text-to-Video Arena
  182. Artificial Analysis Video Editing Arena
  183. LMArena Vision Arena
  184. HourVideoStanford
  185. IDP Leaderboard
  186. OmniAI OCR Leaderboard
  187. Design Arena — 3D DesignIntelligence AI
  188. Design Arena — ASCII Artdesignarena.ai
  189. Design Arena — Android Apps
  190. Design Arena — Data Visualization
  191. Design Arena — Frontend Web Apps
  192. Design Arena — Full-Stack Web Apps
  193. Design Arena — Godot Game Development
  194. Design Arena — Graphic Designdesignarena.ai
  195. Design Arena — HTML Game Development
  196. Design Arena — Image Editing
  197. Design Arena — Image Generation
  198. Design Arena — Image to VideoIntelligence AI
  199. Design Arena — Image to Web App
  200. Design Arena — Image to Website
  201. Design Arena — Logo Design
  202. Design Arena — Multi-Image to Video
  203. Design Arena — Multimodal to Video
  204. Design Arena — Proprietary Harness SlidesIntelligence
  205. Design Arena — Python-PPTX Slides
  206. Design Arena — React Native Apps
  207. Design Arena — SVG Design
  208. Design Arena — Text to Speech
  209. Design Arena — UI Components
  210. Design Arena — Video Editing
  211. Design Arena — Video Generation
  212. Design Arena — Video to Website
  213. Design Arena — Web Design Overall
  214. Design Arena — Website Builders
  215. Design Arena — Website Design
  216. ALL Bench MultimodalFINAL-Bench
  217. BPL OCR Bench
  218. CharadesSTAGao, University of Southern California, Sun, Yang
  219. Do Text Edits Generalize to Visual Generation? Benchmarking Cross-Modal Knowledge Editing in UMMs
  220. FigQAFutureHouse
  221. first bias evaluation dataset for multimodal speech recognition
  222. IDEA-Bench ArenaTongyi Lab
  223. ISL FinePDFs OCR
  224. LOVEU-TGVELOVEU Workshop @ CVPR 2023
  225. MMVet
  226. OCR Bench UFO
  227. OmniInteractCUHK MMLab
  228. OpenScore String Quartets OMRLegato paper, Guang Yang et al
  229. PerceptionTest
  230. SLVMEval
  231. VATEX
  232. VisIT-Bench Multiple Images
  233. VLABench
  234. VLMsAreBlind
  235. WorldArena Track 1
19 not yet confirmedListed in the catalog; no line from its own site or paper confirms it yet.
  1. ParseBench
  2. PerceptionDLM
  3. VideoQA
  4. Biomedical image analysis
  5. BabyVision w/ Python
  6. ZeroBench w/ Python
  7. MMMU-Pro w/ Python
  8. Liquid Extract JSON Validity
  9. Liquid Extract F1
  10. Liquid Extract VLM Judge
  11. MStar
  12. ChatCVQA
  13. AI2D_TEST
  14. VoiceCodeBench
  15. TailorMind
  16. Design Arena — Agentic Game Development
  17. Design Arena — HTML Slides
  18. InferBench Text-to-Image
  19. OMLAB Open Agent Multimodal Leaderboard

Next: pick another domain. Tags come from the RL Research daily scrape, canonicalized nightly.