IHBench - Recovery Quality - Pushback: leaderboard
IHBench (Salimi et al., 2026) interrupts spoken-dialogue models mid-utterance at 428 scripted points in 45 structured-workflow voice conversations across ten domains, with the user's turn given as audio; a GPT-5.4-mini judge (high reasoning) scores the model's next turn, averaged over three epochs. This board is the 105 interruptions in which the user pushes back on what the assistant said, scored by the percentage of turns meeting every recovery criterion; higher is better.
Metric: Recovery pass rate (%; 105 pushbacks, every type-specific criterion met, GPT-5.4-mini judge (high reasoning), audio input, mean of three epochs). Source: arxiv.org. Saturation forecast: Around February 2027. 27 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 2.5 Flash (Thinking) | 75 | #452 (Gemini 2.5 Flash) |
| 2 | Gemini 3.1 Pro (Preview) | 71 | #91 |
| 3 | Qwen3 Omni 30B A3B Instruct | 69 | #902 |
| 4 | Gemini 2.5 Flash (Non-reasoning) | 67 | #452 (Gemini 2.5 Flash) |
| 5 | Gemini 2.5 Pro | 66 | #262 |
| 6 | Voxtral-Small-24B-2507 | 61 | #1291 |
| 7 | Gemma 4 12B (Non-reasoning) | 60 | #927 (Gemma 4 12B) |
| 8 | Gemma 4 12B (Reasoning) | 58 | #927 (Gemma 4 12B) |
| 9 | Qwen2.5-Omni-7B | 49 | #1797 |
| 10 | Phi-4 Multimodal Instruct | 44 | #3914 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=ihbench-recovery-quality-pushback · How It Works · Data refreshed daily, snapshot 2026-11-12.