Tau2-Bench Verified - Retail: leaderboard

The 114 retail tasks of tau2-Bench Verified, as corrected by Amazon AGI (orders, returns, exchanges under the store policy), with gpt-5.1 as the customer; the score is the share of task runs that pass (the README says averaged over 5 runs). Amazon's own Nova 2 rows were added in a later commit that does not state how they were run.

Metric: Retail pass^1 (%). Source: github.com. Saturation forecast: Around February 2027. 14 models tracked. Updated 17 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1Grok 4.1 FastReasoning81.4#429
2Claude Opus 4.580.88#161
3GPT-5.2xHigh79.65#185
4Nova 2 Omni78.3#472
5GPT-5Medium78.25#177
6Gemini 3 Pro77.72#145
7Nova 2 Pro77.7#516
8GPT-5.1High77.54#235
9Claude Sonnet 4.577.19#258
10Nova 2.0 Lite76.5#1125
11GPT-5 MiniMedium73.68#359
12Gemini 2.5 Pro71.26#274
13Claude Haiku 4.569.12#580
14Gemini 2.5 Flash57.72#467

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=tau2-bench-verified-retail · How It Works · Data refreshed daily, snapshot 2026-11-17.