MuLA-Bench - Reasoning: leaderboard

Reasoning questions in MuLA-Bench (Yang et al., 2026), a benchmark of open-ended questions over long in-the-wild recordings, answered from the audio alone, which require an inference from what the recording says or contains rather than general knowledge. Each answer is judged correct or wrong by GPT-5.6 Sol against the reference with rubric and timing checks; accuracy in percent, higher is better.

Metric: Accuracy (%; the 1,146 reasoning questions requiring inference over the recording, both tracks; open-ended questions over in-the-wild long-form recordings (median 36.8 minutes, capped at three hours) in the source language, answered from the full recording without transcripts, audio clipped only to a local model's native input limit; each answer judged correct or not by GPT-5.6 Sol on a route-specific binary rubric with programmatic structural and 3-second timing checks; missing or invalid answers scored wrong). Source: arxiv.org. Saturation forecast: Around February 2027. 10 models tracked.

Top models

No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.

Interactive version: theaggregate.ai/benchmark?slug=mula-bench-reasoning · How It Works · Data refreshed daily, snapshot 2026-11-12.