MMAU-Pro - Instruction Following: leaderboard

Audio-language models answer human-written questions about real-world speech, sound and music clips of up to 10 minutes; the score is accuracy on the 87 spoken format-constraint instructions, checked by regex, from the authors' runs. Unlike the original MMAU, clips are long, can be mixed or multiple, and include spatial audio, voice chat and open-ended questions.

Metric: Accuracy (%). Source: arxiv.org. Saturation forecast: Around February 2027. 18 models tracked. Updated 17 Nov 2026.

Top models

#ModelScoreOverall rank
1Phi-4 Multimodal Instruct65.4#4209
2Qwen2.5-Omni-7B61.3#1883
3Qwen2.5-Omni-3B58.4#3596
4Ming-Lite-Omni-1.548.2#2348
5Baichuan-Omni-1.547.2#3064
6DeSTA2.5-Audio46.5#2076
7R1-AQA44.2#5707
8Audio Reasoner43.4#3028
9DeSTA2-8B-beta41.5
10SALMONN-13B38.5
11Gemma 3n E4B (IT)36.4#2432
12SALMONN-7B33.9
13Audio Flamingo 333.3
14BAT31.8
15GAMA31.7#8292

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=mmau-pro-instruction-following · How It Works · Data refreshed daily, snapshot 2026-11-17.