AA Speech to Speech - Tau-Voice 2.0 - Airline: leaderboard
Sierra tau-bench v2.0.0 voice: 50 English tasks each in airline, retail and telecom; three trials; GPT5.5 low simulated caller speaking ElevenLabs eleven_v3; common prompt/tools/policy, 2400-second call limit. State/action/output checks; infrastructure errors excluded and rerun. Overall is equal domain mean. GPT-Live's source-declared backend prompt differs. Separate from V1; no cross-edition score replacement. Metric: Pass@1 (%), mean of three trials. Source: https://artificialanalysis.ai/speech-to-speech
Metric: Pass@1 (%), mean of three trials. Source: artificialanalysis.ai. Saturation forecast: Rough model projection: around 2026. 11 models tracked. Updated 17 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | GPT-Live 1 (Astra, medium) | 84 | ||
| 2 | Phonic 1 | High | 83.33 | |
| 3 | GPT-Live 1 (Sol, low) | 83.33 | ||
| 4 | Grok Voice 3.0 | 71.33 | ||
| 5 | Qwen Audio 3.0 Realtime Plus | 64.67 | ||
| 6 | Eleven Realtime v1 Max | 63.33 | ||
| 7 | Gemini 3.8 Live (Extended Thinking) | High | 62.67 | |
| 8 | GPT Realtime 2.1 | High | 60 | |
| 9 | Gemini 3.8 Live | 58 | ||
| 10 | Gemini 3.1 Flash Live High | 54.67 | ||
| 11 | Eleven Realtime v1 Mini | 50 |
Interactive version: theaggregate.ai/benchmark?slug=aa-speech-to-speech-tau-voice-2-0-airline · How It Works · Data refreshed daily, snapshot 2026-11-17.