Domain on the map · as Media
Audio: RL environments and AI benchmarks
Speech and sound: transcribing, understanding and generating audio.
- RL environments
- 1
- AI benchmarks
- 45
- Shares entries with
- 2 categories
Snapshot 2026-11-16. How categories are built · Where the data comes from
In short
- What are RL environments for audio?
- A simulated workplace where an AI agent practises a task (a code repository, a browser, business software) and an automatic grader scores each attempt. Labs train on thousands of these attempts. Bought to improve models. For audio, that means speech and sound: transcribing, understanding and generating audio.
- Who builds RL environments for audio?
- Uber AI.
- What are the main audio benchmarks?
- HEAR, LatencyGrid, Medical STT Benchmark, SpeakerSleuth, AGL1K and Audio Reasoning Challenge, among 42 confirmed below.
RL environments 1
-
Uber AI
No line from its own site or paper places it in Audio yet; the catalog lists it here.
AI benchmarks 45
-
HEAR
Tests media“HEAR evaluates audio representations using a benchmark suite across a variety of domains, including speech, environmental sound, and music.” source ↗
-
LatencyGrid
Tests media“Total end-to-end latency for common STT + LLM + TTS combinations.” source ↗
-
Medical STT Benchmark
Tests media“The latest speech models, tested on the same medical audio. Compare batch and realtime transcription, including how quickly the text arrives.” source ↗
-
SpeakerSleuth
Tests media“a benchmark evaluating whether LALMs can reliably judge speaker consistency across multi-turn dialogues through three tasks reflecting real-world requirements” source ↗
-
AGL1K
Tests media“we introduce AGL1K, the first audio geo-localization benchmark for audio language models (ALMs), spanning 72 countries and territories.” source ↗
-
Audio Reasoning Challenge
Tests media“a 1,000-item dataset designed for deep audio reasoning across speech, sound, music, and mixed-modality scenarios” source ↗
-
PALM-Bench
Tests media“we create the first benchmark (PALM-Bench) to foster the methodological advances in PALM and enable structured evaluation on several tasks across multi-speaker scenarios.” source ↗
-
PodBench
Tests media“PodBench offers a reproducible testbed to address these challenges in long-form, audio-centric generation.” source ↗
-
SonicBench
Tests media“SonicBench is a **psychophysically grounded** benchmark that probes **physical audio perception** rather than semantics” source ↗
-
From Sounds to Scenes
Tests media“we introduce the CASU benchmark, which assesses whether Audio LLMs can interpret auditory scenes composed of speech, acoustic events (e.g., announcements), and background environments” source ↗
-
AEQ-Bench
Tests media“judging the empathy of audio responses without relying on text transcription” source ↗
-
Afrispeech Semantics
Tests media“We evaluate audio language models across five semantic and paralinguistic reasoning tasks: entailment, consistency, plausibility, accent drift, and accent restraint.” source ↗
-
Artificial Analysis Speech-to-Text
Tests media“Measures transcription accuracy across 3 datasets to evaluate models in real-world speech with diverse accents, domain-specific language, and challenging channel & acoustic conditions.” source ↗
-
ASR-FairBench
Tests media“we introduce the ASR-FAIRBENCH leaderboard which is designed to assess both the accuracy and equity of ASR models in real-time.” source ↗
-
BCoughBench
Tests media“We introduce BCoughBench, evaluating five FMs (OPERA-CT/CE/GT, HeAR, M2D+Resp) on nine classification tasks” source ↗
-
VoxParadox
Tests media“we introduce VoxParadox, an adversarial benchmark with 2,000 verified examples, spanning 10 paralinguistic tasks, created with controlled speech synthesis” source ↗
-
AHa-Bench
Tests media“AHa-Bench comprises three distinct audio types: Speech, Sound, and Music.” source ↗
-
ambiguous emotion recognition benchmark for ALMs with TTS
Tests media“we introduce the first benchmark for ambiguous emotion recognition in speech with ALMs under test-time scaling.” source ↗
-
AOR-Bench
Tests media“the first benchmark for over-refusal specifically designed for LALMs. AOR-Bench contains 3,000 pseudo-harmful audio samples across six scenario categories.” source ↗
-
CBT-Audio
Tests media“We evaluate 10 open source audio language models under three input conditions, where models receive only patient audio, only the transcript, or both audio and transcript.” source ↗
-
HearSay Benchmark: Do Audio LLMs Leak What They Hear?
Tests media“investigate whether ALLMs inadvertently leak user privacy solely through acoustic voiceprints and introduces \textit{HearSay}HearSay, a comprehensive benchmark constructed from over 22,000 real-world audio clips” source ↗
-
MCGA
Tests media“a 119-hour corpus comprising 22,000 audio samples. It encompasses a diverse range of literary genres across six tasks: Automatic Speech Recognition (ASR)” source ↗
-
MusTBENCH
Tests media“we introduce MusTBENCH, a music-expert-validated benchmark designed to evaluate temporal grounding in LALMs through five temporally grounded question-answering tasks.” source ↗
-
ParaPairAudioBench
Tests media“We introduce ParaPairAudioBench, a pairwise benchmark of 5,175 audio pairs across five paralinguistic dimensions: Style, Rate, Emphasis, Age, and Gender.” source ↗
-
PitchBench
Tests media“We introduce PitchBench, an evaluation suite that systematically measures pitch hearing in ALMs.” source ↗
-
ReasonAudio
Tests media“we introduce ReasonAudio, the first reasoning-intensive benchmark for Text-Audio Retrieval, comprising 1,000 queries and 10,000 composite audio clips” source ↗
-
SpurAudio
Tests media“We introduce SpurAudio, a benchmark that leverages the natural separability of foreground events and background environments in audio” source ↗
-
UltraEval-Audio
Tests media“we introduce UltraEval-Audio, a unified evaluation framework for audio foundation models, specifically designed for both audio understanding and generation tasks.” source ↗
-
Bias in the Ear of the Listener
Tests media“BiasInEar is a benchmark for evaluating speech bias in multilingual multimodal large language models (MLLMs).” source ↗
-
Artificial Analysis Instrumental Music Arena
Tests media“AA-Music-Instrumental v1.1 ranks models generating music without vocals” source ↗
-
Artificial Analysis Vocals Music Arena
Tests media“AA-Music-Vocal v1.1 ranks models that generate music with singing or vocal elements.” source ↗
-
Artificial Analysis Speech Arena — Provider Voices
Tests media“Compare Text to Speech (TTS) models using each provider's own native voices.” source ↗
-
Speech Arena
Tests media“Best text to speech models, ranked by crowd-sourced blind listening tests.” source ↗
-
TurnBench
Tests media“A multi-domain benchmark for evaluating conversational turn-taking. We hand-annotate end-of-turn and interruption events in dual-channel human conversations” source ↗
-
A Closer Look at Failure Modes in Temporal Understanding of Large Audio-Language Models
Tests media“we introduce a benchmark with 1,657 questions across three foundational tasks designed specifically for mechanistic analysis.” source ↗
-
ALME Benchmark
Tests media“ALME evaluates how audio-LLMs handle conflicting audio and text inputs across **8 typologically diverse languages**” source ↗
-
Open ASR Leaderboard
Tests media“We present the Open ASR Leaderboard, a reproducible benchmarking platform with community contributions from academia and industry.” source ↗
-
SAFE Audio Challenge Task 1
Tests media“a fully blind evaluation framework designed to benchmark detection models across progressively harder scenarios: raw synthetic speech, processed audio (e.g., compression, resampling)” source ↗
-
SAFE Audio Challenge Task 2
Tests media“a fully blind evaluation framework designed to benchmark detection models across progressively harder scenarios: raw synthetic speech, processed audio (e.g., compression, resampling)” source ↗
-
SAFE Audio Challenge Task 3
Tests media“a fully blind evaluation framework designed to benchmark detection models across progressively harder scenarios: raw synthetic speech, processed audio (e.g., compression, resampling), and laundered audio” source ↗
-
Voice Benchmark
Tests media“Open real-world benchmark for voice AI platforms” source ↗
-
When Audio-Language Models Fail to Leverage Multimodal Context for Dysarthric Speech Recognition
Tests media“tests whether diagnosis labels, clinician-derived speech ratings, and progressively richer clinical descriptions improve transcription accuracy for dysarthric speech” source ↗
3 not yet confirmedListed in the catalog; no line from its own site or paper confirms it yet.
Data 11
- Contra Labs
- Exabite
- Taste Labs
-
Verita AI
Sells data“Our platform solves this by collecting open-ended information about every expert in our community and building on each profile as they work with us.” source ↗
Covers media“We care about aesthetics in all its nuance, working on how AI understands taste and how to align it with the people it serves.” source ↗
- Wirestock
- AfterQuery
-
Refresh
Sells data“Refresh builds environments by partnering with expert engineers in our network to source real tasks that break the leading models.” source ↗
Covers media“Agents working inside Blender, from building scenes and materials to rigging characters, keyframing animation, and running cloth and physics simulations.” source ↗
- Revelo
- Scale AI
- Surge AI
- Vetto AI
Next: pick another domain. Tags come from the RL Research daily scrape, canonicalized nightly.