Domain on the map · as Media
Creative AI benchmarks and RL environments
Creative and aesthetic work, such as images, style and taste, where output is judged by people rather than by an automatic check.
- RL environments
- 3
- AI benchmarks
- 8
- Shares entries with
- 1 category
Snapshot 2026-11-05. How categories are built · Where the data comes from
In short
- What are RL environments for creative work?
- A simulated workplace where an AI agent practises a task (a code repository, a browser, business software) and an automatic grader scores each attempt. Labs train on thousands of these attempts. Bought to improve models. For creative work, that means creative and aesthetic work, such as images, style and taste, where output is judged by people rather than by an automatic check.
- Who builds RL environments for creative work?
- Taste Labs, Contra Labs and Verita AI.
- What are the main creative AI benchmarks?
- NeoEvalPlusN, TASTE, IDEAFix, NarrativeWorldBench, QUIET and RP-Bench, among 7 confirmed below.
RL environments 3
-
Taste Labs
Covers media“We’re making the unverifiable verifiable, starting with design.” source ↗
- Contra Labs
-
Verita AI
Covers media“We care about aesthetics in all its nuance, working on how AI understands taste and how to align it with the people it serves.” source ↗
AI benchmarks 8
-
NeoEvalPlusN
Tests media“PS=creative writing” source ↗
-
TASTE
Tests media“we benchmark preference models on TASTE and find that off-the-shelf VLM judges and dedicated T2I scorers fail to reach majority agreement with the designer panel” source ↗
-
IDEAFix
No line from its own site or paper places it in Creative yet; the catalog lists it here.
-
NarrativeWorldBench
Tests media“We introduce NarrativeWorldBench, an open benchmark of nine narrative-structure metrics evaluated across horizons h in {10, 20, 50, 100, 200}” source ↗
-
QUIET
Tests media“a diagnostic benchmark for LLM creative capability based on multi-blank cascaded story cloze” source ↗
-
RP-Bench
Tests media“not just writing quality, but character consistency, user agency respect, lorebook integration, temporal reasoning, and genre-specific craft.” source ↗
-
Ex Ante Evaluation of AI-Induced Idea Diversity Collapse
Tests media“Across short stories, marketing slogans, and alternative-uses tasks, three frontier LLMs fall below parity across crowding kernels.” source ↗
1 not yet confirmedListed in the catalog; no line from its own site or paper confirms it yet.
Data 14
- Contra Labs
- Exabite
- Taste Labs
-
Verita AI
Sells data“demand has shifted from large, distributed global workforces to cultivated, well-coordinated cohorts of PhD-level subject-matter experts and consultants who don’t need gig work.” source ↗
Covers media“We care about aesthetics in all its nuance, working on how AI understands taste and how to align it with the people it serves.” source ↗
- Wirestock
- AfterQuery
- Cua
- Markov
- Refresh
- Revelo
- Rise Data Labs
- Surge AI
- Symbal
- Vetto AI
Next: pick another domain. Tags come from the RL Research daily scrape, canonicalized nightly.