KAware - Internal Function Pass Rate: leaderboard
End-to-end task success on KAware (2026) tasks solvable from parametric knowledge although tools are offered: the share of tasks whose trajectory and final answer an LLM judge marks correct, in percent; higher is better. KAware (2026) keeps tools available on every task, so an agent must judge whether it needs them.
Metric: The end-to-end pass rate (%): share of tasks whose trajectory and final answer a gpt-4.1 judge marks correct, on the internal function setting, on the KAware tool-use tasks, where tools are always available but the task may need them (external function, 310 tasks), mix them with parametric knowledge (hybrid composition, 368 tasks) or be solvable from knowledge alone (internal function, 398 tasks); higher is better. Source: arxiv.org. Saturation forecast: Not forecast. 18 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4.5 (Thinking) | 96.23 | #250 (Claude Sonnet 4.5) |
| 2 | Qwen 3 Max (Preview) | 95.98 | #447 |
| 3 | Qwen 3 235B A22B 2507 (Thinking) | 93.22 | #463 |
| 4 | Qwen 3 235B A22B 2507 Instruct | 90.45 | #619 |
| 5 | Claude Sonnet 4.5 | 87.94 | #250 |
| 6 | GPT-5.5 | 86.68 | #49 |
| 7 | DeepSeek V3.2 (Non-reasoning) | 86.18 | #406 (DeepSeek V3.2) |
| 8 | Qwen 3.5 397B A17B | 84.92 | #246 |
| 9 | Qwen 3.5 397B A17B (Non-reasoning) | 84.67 | #246 (Qwen 3.5 397B A17B) |
| 10 | DeepSeek V4 Flash (Non-reasoning) | 83.42 | #258 (DeepSeek V4 Flash) |
| 11 | GPT-5 | 60.8 | #167 |
| 12 | O4 Mini (2025-04-16) | 60.05 | #365 |
| 13 | GPT-4o (2024-11-20) | 59.3 | #866 |
| 14 | GPT-4.1 | 59.05 | #495 |
| 15 | Gemini 3 Flash (Preview) | 59.05 | #144 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=kaware-internal-function-pass-rate · How It Works · Data refreshed daily, snapshot 2026-11-13.