GTA-Workflow - Tool Success Rate: leaderboard

Tool call validity on GTA-Workflow, the open-ended tier of GTA-2 (2026): 132 long-horizon productivity workflows from real user queries with deployed tools and multimodal inputs, run end to end in a ReAct agent framework and judged by GPT-5.2 against recursive checkpoint rubrics: the share of tool invocations without system errors or syntax violations. Percent; higher is better.

Metric: Tool success rate (%): share of tool invocations without system errors or syntax violations on GTA-Workflow, the open-ended long-horizon workflow tier of GTA-2 (132 real-user tasks with deployed tools and multimodal inputs), run end to end in the default Lagent ReAct framework; deliverables judged by GPT-5.2 against recursive checkpoint rubrics; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 13 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 2.5 Pro91.2#280
2Kimi K289.85#448
3Qwen 3 235B A22B88.98#633
4DeepSeek V3.288.81#425
5Claude Sonnet 4.588.02#267
6Llama 4 Scout87.74#1825
7Grok 487.47#321
8GPT-587.31#185
9Llama 3.1 70B Instruct28.71#1579
10Qwen 3 8B16.97#1890
11Llama 3.1 8B Instruct13.44#4655
12Qwen 3 30B A3B1.94#1165
13Llama 3.2 3B Instruct0.1#7311

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=gta-workflow-tool-success-rate · How It Works · Data refreshed daily, snapshot 2026-11-22.