HumDial-EIBench - Acoustic-Semantic Conflict (English): leaderboard

Acoustic-semantic conflict on HumDial-EIBench (2026): 50 single-turn English human recordings whose vocal emotion contradicts the literal sentiment of the words, where the model must identify the true emotional state of the speaker among four options. Accuracy in percent; higher is better.

Metric: Accuracy (%) of identifying the speaker's true emotional state on 50 single-turn English conflict samples, where the speaker's vocal emotion contradicts the literal text sentiment and a literal-text option is the main distractor (four options, human-recorded); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 8 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 2.5 Flash62#463
2GPT-4o Audio46#1129
3Doubao-realtime40
4Kimi-Audio38#1735
5Qwen2.5-Omni32#1588
6Step-Audio-2-mini26
7Freeze-Omni22
8GLM-4-Voice20

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=humdial-eibench-acoustic-semantic-conflict-english · How It Works · Data refreshed daily, snapshot 2026-11-21.