HumDial-EIBench - Acoustic-Semantic Conflict (Chinese): leaderboard

Acoustic-semantic conflict on HumDial-EIBench (2026): 50 single-turn Chinese human recordings whose vocal emotion contradicts the literal sentiment of the words, where the model must identify the true emotional state of the speaker among four options. Accuracy in percent; higher is better.

Metric: Accuracy (%) of identifying the speaker's true emotional state on 50 single-turn Chinese conflict samples, where the speaker's vocal emotion contradicts the literal text sentiment and a literal-text option is the main distractor (four options, human-recorded); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 8 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Doubao-realtime54
2Gemini 2.5 Flash50#463
3Kimi-Audio48#1735
4GPT-4o Audio34#1129
5GLM-4-Voice30
6Step-Audio-2-mini24
7Qwen2.5-Omni22#1588
8Freeze-Omni18

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=humdial-eibench-acoustic-semantic-conflict-chinese · How It Works · Data refreshed daily, snapshot 2026-11-21.