ToolMaze (Failure-Aware Prompt): leaderboard
ToolMaze (2026) tests tool-using LLM agents on 400 DAG-structured tasks over 270 tools, each also run with injected tool failures that are explicit or implicit and transient or permanent. The board is the composite of task success, recovery rate and one minus recovery cost, in percent, under a prompt that warns of tool failures and outlines recovery strategies; higher is better.
Metric: Composite score (%): mean of task success over the clean and four perturbed settings, perturbation recovery rate, and one minus recovery cost, each macro-averaged over the four topology levels; 400 DAG-structured tool-use tasks (four topology levels) over 270 tools, each also run under four injected tool-failure modes (explicit or implicit, transient or permanent); sandboxed, 25 agent steps, temperature 1, a failure-aware prompt that warns of tool failures and outlines recovery strategies (the clean runs keep the standard prompt); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) | 52.95 | #94 |
| 2 | GPT-5.5 | 49.96 | #49 |
| 3 | Claude Sonnet 4.6 | 46.14 | #161 |
| 4 | DeepSeek V4 Pro | 45.09 | #188 |
| 5 | GLM-5.1 | 38.14 | #179 |
| 6 | Qwen 3.5 397B A17B | 35.56 | #246 |
| 7 | Qwen 3.6 27B | 34.13 | #341 |
| 8 | Qwen 3.5 35B A3B | 32.56 | #482 |
| 9 | MiniMax-M2.7 | 27.5 | #518 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=toolmaze-failure-aware-prompt · How It Works · Data refreshed daily, snapshot 2026-11-13.