When2Call (Voice): leaderboard
The When2Call benchmark of deciding whether a request should trigger a tool call, with 300 non-multiple-choice queries spoken instead of typed. From Text to Voice (Laskar et al., Dialpad, 2026) turns text tool-calling benchmarks into speech by synthesizing every user query with three TTS systems in a female and a male voice, keeping the tool schemas, labels and scorers; omni-modal models answer from the audio. F1 in percent of the call versus no-call decision, averaged over the six voice configurations; higher is better.
Metric: F1 (%; tool call versus no tool call, parsed from the response; 300 spoken non-multiple-choice When2Call queries, mean over six TTS voice configurations). Source: arxiv.org. Saturation forecast: Around February 2027. 7 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen3 Omni 30B A3B Instruct | 60.4 | #902 |
| 2 | Phi-4 Multimodal Instruct | 53.6 | #3914 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=when2call-voice · How It Works · Data refreshed daily, snapshot 2026-11-12.