ToolMaze (Failure-Aware Prompt) - Perturbed Task Success: leaderboard
ToolMaze (2026) tests tool-using LLM agents on 400 DAG-structured tasks over 270 tools, each also run with injected tool failures that are explicit or implicit and transient or permanent. This board is the task success rate in percent averaged over the four failure modes, under a prompt that warns of tool failures and outlines recovery strategies; higher is better.
Metric: Task success rate (%) averaged over the four tool-failure modes, macro-averaged over the four topology levels; 400 DAG-structured tool-use tasks (four topology levels) over 270 tools, each also run under four injected tool-failure modes (explicit or implicit, transient or permanent); sandboxed, 25 agent steps, temperature 1, a failure-aware prompt that warns of tool failures and outlines recovery strategies (the clean runs keep the standard prompt); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) | 46.25 | #94 |
| 2 | GPT-5.5 | 43.56 | #49 |
| 3 | Claude Sonnet 4.6 | 42 | #161 |
| 4 | DeepSeek V4 Pro | 40.81 | #188 |
| 5 | GLM-5.1 | 33.75 | #179 |
| 6 | Qwen 3.6 27B | 23.94 | #341 |
| 7 | Qwen 3.5 397B A17B | 22.06 | #246 |
| 8 | MiniMax-M2.7 | 21.94 | #518 |
| 9 | Qwen 3.5 35B A3B | 19.31 | #482 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=toolmaze-failure-aware-prompt-perturbed-task-success · How It Works · Data refreshed daily, snapshot 2026-11-13.