KAware - External Function: leaderboard
The knowing-acting score of KAware (2026) on tasks that can only be solved with external tools: the harmonic mean of how well the agent states the needed tools and how well its actual tool calls match them, in percent; higher is better. KAware (2026) keeps tools available on every task, so an agent must judge whether it needs them.
Metric: The knowing-acting score (%, KAS): harmonic mean of knowing accuracy (Jaccard overlap between the tools the agent says the task needs and the gold tool set) and acting accuracy (Jaccard overlap between the tools it actually calls and the gold set), on the external function setting, on the KAware tool-use tasks, where tools are always available but the task may need them (external function, 310 tasks), mix them with parametric knowledge (hybrid composition, 368 tasks) or be solvable from knowledge alone (internal function, 398 tasks); higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 18 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3 235B A22B 2507 (Thinking) | 97.42 | #463 |
| 2 | Qwen 3 Max (Preview) | 97.36 | #447 |
| 3 | Qwen 3.5 397B A17B | 97.34 | #246 |
| 4 | Claude Sonnet 4.5 (Thinking) | 97.3 | #250 (Claude Sonnet 4.5) |
| 5 | Qwen 3.5 397B A17B (Non-reasoning) | 96.62 | #246 (Qwen 3.5 397B A17B) |
| 6 | DeepSeek V4 Flash (Non-reasoning) | 96 | #258 (DeepSeek V4 Flash) |
| 7 | GPT-5.5 | 95.9 | #49 |
| 8 | Claude Sonnet 4.5 | 95.62 | #250 |
| 9 | Gemini 3 Pro (Preview) | 91.75 | #113 |
| 10 | GPT-4.1 | 91.26 | #495 |
| 11 | GPT-4o (2024-11-20) | 90.98 | #866 |
| 12 | DeepSeek V3.2 (Non-reasoning) | 90.84 | #406 (DeepSeek V3.2) |
| 13 | Gemini 3 Flash (Preview) | 89.61 | #144 |
| 14 | O4 Mini (2025-04-16) | 89.22 | #365 |
| 15 | GPT-5 | 88.36 | #167 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=kaware-external-function · How It Works · Data refreshed daily, snapshot 2026-11-13.