ToolFailBench: leaderboard
Metric: Clean tool-use rate (%; share of the 750 tool-required single-turn tasks whose trace calls the needed tool and answers with the returned value, without skipping the tool, ignoring its result or fabricating output; labels by majority vote of a rule classifier and two LLM judges; mock tool returns contradict plausible prior values; temperature 0, 1,024-token output limit; higher is better). Source: arxiv.org. Saturation forecast: Around December 2026. 19 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | Grok 4.3 | 86.33 |
| 2 | Grok 4.1 Fast (Reasoning) | 84.11 |
| 3 | Qwen 2.5 32B Instruct | 82.68 |
| 4 | Qwen 3.6 27B | 79.33 |
| 5 | Claude Sonnet 4.5 | 79.28 |
| 6 | GPT-5.4 Mini | 79.14 |
| 7 | QwQ-32B | 79.04 |
| 8 | Qwen 2.5 72B Instruct | 79 |
| 9 | Qwen 3.6 35B A3B | 78.47 |
| 10 | Gemma 4 31B (IT) | 78.12 |
| 11 | Qwen 3.5 27B | 77.38 |
| 12 | Claude Haiku 4.5 | 76.47 |
| 13 | DeepSeek V4 Flash | 75.84 |
| 14 | GLM-4.7 Flash | 71.49 |
| 15 | Qwen 3.5 9B | 70.03 |
Interactive version: theaggregate.ai/benchmark?slug=toolfailbench · How It Works · Data refreshed daily, snapshot 2026-09-29.