Tau2-Bench Verified - Airline: leaderboard
The 50 airline tasks of tau2-Bench Verified, as corrected by Amazon AGI (bookings, changes, cancellations, compensation under the airline policy), with gpt-5.1 as the customer; the score is the share of task runs that pass (the README says averaged over 5 runs). Amazon's own Nova 2 rows were added in a later commit that does not state how they were run.
Metric: Airline pass^1 (%). Source: github.com. Saturation forecast: Around February 2027. 14 models tracked. Updated 17 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Claude Opus 4.5 | 74.4 | #161 | |
| 2 | GPT-5.2 | xHigh | 74.27 | #185 |
| 3 | GPT-5.1 | High | 72.4 | #235 |
| 4 | Grok 4.1 Fast | Reasoning | 72 | #429 |
| 5 | GPT-5 | Medium | 72 | #177 |
| 6 | Gemini 3 Pro | 70.8 | #145 | |
| 7 | GPT-5 Mini | Medium | 68.8 | #359 |
| 8 | Nova 2 Omni | 68.8 | #472 | |
| 9 | Claude Sonnet 4.5 | 66.8 | #258 | |
| 10 | Nova 2 Pro | 65.2 | #516 | |
| 11 | Nova 2.0 Lite | 64.8 | #1125 | |
| 12 | Gemini 2.5 Pro | 60 | #274 | |
| 13 | Claude Haiku 4.5 | 54 | #580 | |
| 14 | Gemini 2.5 Flash | 44 | #467 |
No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.
Interactive version: theaggregate.ai/benchmark?slug=tau2-bench-verified-airline · How It Works · Data refreshed daily, snapshot 2026-11-17.