Toolathlon Verified Pass@1 (Anthropic Internal Harness): leaderboard
Mean single-trial success over 108 Toolathlon-Verified tasks, with three trials per task in Anthropic’s internal harness at max effort. Execution-based checks assess tool use across 32 applications. Higher is better; model labels retain documented classifier-fallback conditions. Source: Claude Opus 5.5 system card. Distinct from pass@3 and pass^3.
Metric: Pass@1 (%). Source: www-cdn.anthropic.com. Saturation forecast: Around September 2027. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 5 (Max) | 80.6 | #12 (Claude Opus 5) |
| 2 | Claude Opus 4.8 (Max) | 79.9 | #38 (Claude Opus 4.8) |
| 3 | Claude Opus 5.5 (Max) | 77.8 | #3 (Claude Opus 5.5) |
| 4 | Claude Sonnet 5 (Max) | 74.7 | #55 (Claude Sonnet 5) |
No result here: #5 GPT-6 Astra, #7 Gemini 4 Argon, #8 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=toolathlon-verified-pass-1-anthropic-internal-harness · How It Works · Data refreshed daily, snapshot 2026-10-16.