Confetti (Voice): leaderboard
The Confetti tool-calling benchmark with its 313 tool-requiring queries spoken instead of typed. From Text to Voice (Laskar et al., Dialpad, 2026) turns text tool-calling benchmarks into speech by synthesizing every user query with three TTS systems in a female and a male voice, keeping the tool schemas, labels and scorers; omni-modal models answer from the audio. AST soft accuracy in percent (exact match on the function name and non-string arguments, AlignScore on string arguments), averaged over the six voice configurations; higher is better.
Metric: AST soft accuracy (%; exact match on function name and non-string arguments, AlignScore on string arguments; 313 spoken Confetti queries, mean over six TTS voice configurations). Source: arxiv.org. Saturation forecast: Around February 2027. 7 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen3 Omni 30B A3B Instruct | 60.4 | #902 |
| 2 | Phi-4 Multimodal Instruct | 23.3 | #3914 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=confetti-voice · How It Works · Data refreshed daily, snapshot 2026-11-12.