tau-Voice (Clean) - Airline: leaderboard
Metric: pass@1 (%): share of tasks whose final database state matches the goal (agent communications checked by an LLM evaluator), full-duplex voice agent talking to a GPT-4.1 voice user simulator (ElevenLabs v3 speech) under the Clean condition (clear American-accented speech, G.711 telephony, no noise or interruptions), on the 50 airline tasks (flight changes, cancellations, upgrades, bookings); higher is better. Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 3 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT Realtime 1.5 | 48 | |
| 2 | Grok Voice Agent | 46 | |
| 3 | Gemini Live 2.5 Flash Native Audio | 28 |
Interactive version: theaggregate.ai/benchmark?slug=tau-voice-clean-airline · How It Works · Data refreshed daily, snapshot 2026-10-11.