MMAU-Pro - Voice Chat: leaderboard

Audio-language models answer human-written questions about real-world speech, sound and music clips of up to 10 minutes; the score is accuracy on the 290 spoken STEM, prosody and world-knowledge questions, from the authors' runs. Unlike the original MMAU, clips are long, can be mixed or multiple, and include spatial audio, voice chat and open-ended questions.

Metric: Accuracy (%). Source: arxiv.org. Saturation forecast: Estimated already saturated. 18 models tracked. Updated 17 Nov 2026.

Top models

#ModelScoreOverall rank
1Qwen2.5-Omni-7B60#1883
2Audio Flamingo 358.6
3Gemma 3n E4B (IT)58.3#2432
4DeSTA2-8B-beta54.8
5SALMONN-13B53.2
6Gemma 3n E2B (IT)51.4#4283
7DeSTA2.5-Audio51#2076
8Qwen2.5-Omni-3B46.5#3596
9Ming-Lite-Omni-1.544.5#2348
10Audio Reasoner43.4#3028
11Phi-4 Multimodal Instruct42.7#4209
12Baichuan-Omni-1.540#3064
13Audio Flamingo 237.2
14SALMONN-7B36.5
15R1-AQA32.7#5707

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=mmau-pro-voice-chat · How It Works · Data refreshed daily, snapshot 2026-11-17.