Tau2-Bench Verified - Telecom: leaderboard

The 114 base telecom tasks of tau2-Bench Verified, in which the agent troubleshoots a phone line while the simulated customer (gpt-5.1) acts on their own device; the score is the share of task runs that pass (the README says averaged over 5 runs). Amazon's own Nova 2 rows were added in a later commit that does not state how they were run.

Metric: Telecom pass^1 (%). Source: github.com. Saturation forecast: Estimated already saturated. 14 models tracked. Updated 17 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1Grok 4.1 FastReasoning94.74#429
2Nova 2 Pro92.7#516
3Claude Opus 4.590.7#161
4Gemini 3 Pro89.65#145
5GPT-5Medium89.5#177
6GPT-5.2xHigh86.99#185
7GPT-5.1High80.53#235
8Nova 2 Omni80#472
9Nova 2.0 Lite76#1125
10Claude Sonnet 4.575.96#258
11GPT-5 MiniMedium67.02#359
12Gemini 2.5 Pro37.89#274
13Claude Haiku 4.537.19#580
14Gemini 2.5 Flash22.98#467

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=tau2-bench-verified-telecom · How It Works · Data refreshed daily, snapshot 2026-11-17.