τ³-bench-LILT - German - Retail - Pass^1: leaderboard
τ³-bench-LILT - German - Retail - Pass^1: Pass^1 (fraction) in LILT Aurora's localized evaluation. Localized customer-support tool use under the official tau3 scaffold; 4 trials and domain-specific judges; banking uses a 20-task sample.
Metric: Pass^1 (fraction). Source: aurora.lilt.com. Saturation forecast: Rough model projection: around 2026. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 5.5 (Aurora Run: τ³-bench-LILT; effort low; OpenRouter) | 0.94 | |
| 2 | GPT-6 Sol (Aurora Run: τ³-bench-LILT; effort none; OpenRouter) | 0.93 | |
| 3 | Claude Opus 5 (Aurora Run: τ³-bench-LILT; effort off; OpenRouter) | 0.92 | |
| 4 | GPT-5.6 Sol (Aurora Run: τ³-bench-LILT; effort none; OpenRouter) | 0.91 | |
| 5 | Nemotron 3 Ultra (Aurora Run: τ³-bench-LILT; settings unreported) | 0.9 | |
| 6 | Gemini 3.8 Flash (Aurora Run: τ³-bench-LILT; effort low; OpenRouter) | 0.9 | |
| 7 | Mistral Medium 3.5 (Aurora Run: τ³-bench-LILT; effort off; local vLLM; mistralai/Mistral-Medium-3.5-128B; temperature 0.1) | 0.82 | |
| 8 | Command A+ (Aurora Run: τ³-bench-LILT; effort no off switch; local vLLM; CohereLabs/command-a-plus-05-2026) | 0.71 |
Interactive version: theaggregate.ai/benchmark?slug=tau3-bench-lilt-german-retail-pass-pow-1 · How It Works · Data refreshed daily, snapshot 2026-11-05.