HalluAudio - Speech: leaderboard

Hallucination detection in the speech domain of HalluAudio (2026): human-verified audio question-answer pairs with adversarial prompts, mixed audio and unanswerable questions probe whether an audio-language model reports sounds or content that are not there. Unweighted mean accuracy over ten task types, in percent; higher is better.

Metric: Average classification accuracy (%) over the ten speech task types of HalluAudio (human-verified audio question-answer pairs with adversarial, mixed-audio and unanswerable prompts; zero-shot; outputs normalized to labels, refusals counted wrong); unweighted mean of the task accuracies; higher is better. Source: arxiv.org. Saturation forecast: Around 2032. 11 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1MiMo-Audio-7B-Instruct52.84#1336
2GPT-4o Audio (Preview)45.51#793
3LLaMA-Omni2-7B40.94
4Kimi-Audio-7B40.73
5Step-Audio-2-mini38.29
6Phi-4 Multimodal Instruct35.47#4154
7Qwen-Audio-Chat32.16
8Qwen2-Audio-7B28.72
9Llama 3.1 8B Omni18.26
10Pengi16.72
11Gemini 2.5 Flash16.6#463

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=halluaudio-speech · How It Works · Data refreshed daily, snapshot 2026-11-21.