Tau2-Bench Verified - Telecom: leaderboard
The 114 base telecom tasks of tau2-Bench Verified, in which the agent troubleshoots a phone line while the simulated customer (gpt-5.1) acts on their own device; the score is the share of task runs that pass (the README says averaged over 5 runs). Amazon's own Nova 2 rows were added in a later commit that does not state how they were run.
Metric: Telecom pass^1 (%). Source: github.com. Saturation forecast: Estimated already saturated. 14 models tracked. Updated 17 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Grok 4.1 Fast | Reasoning | 94.74 | #429 |
| 2 | Nova 2 Pro | 92.7 | #516 | |
| 3 | Claude Opus 4.5 | 90.7 | #161 | |
| 4 | Gemini 3 Pro | 89.65 | #145 | |
| 5 | GPT-5 | Medium | 89.5 | #177 |
| 6 | GPT-5.2 | xHigh | 86.99 | #185 |
| 7 | GPT-5.1 | High | 80.53 | #235 |
| 8 | Nova 2 Omni | 80 | #472 | |
| 9 | Nova 2.0 Lite | 76 | #1125 | |
| 10 | Claude Sonnet 4.5 | 75.96 | #258 | |
| 11 | GPT-5 Mini | Medium | 67.02 | #359 |
| 12 | Gemini 2.5 Pro | 37.89 | #274 | |
| 13 | Claude Haiku 4.5 | 37.19 | #580 | |
| 14 | Gemini 2.5 Flash | 22.98 | #467 |
No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.
Interactive version: theaggregate.ai/benchmark?slug=tau2-bench-verified-telecom · How It Works · Data refreshed daily, snapshot 2026-11-17.