KOPA-Bench - Pass@1: leaderboard
KOPA-Bench evaluates multi-step tool use over Korean public APIs on 145 tasks, with four rollouts per task. Pass@1 averages single-attempt success, pass@4 measures at least one success, and Action measures tool-call accuracy against golden actions. The paper's GRPO-tuned models remain distinct from their base checkpoints. This series reports KOPA-Bench - Pass@1 from 2609.05395v1, S3.T1: Pass@1 (%). Higher is better.
Metric: Pass@1 (%). Source: arxiv.org. Saturation forecast: Around February 2027. 11 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4.6 | 46.55 | #97 |
| 2 | Qwen 3.5 27B | 44.82 | #194 |
| 3 | GPT-5.1 | 37.06 | #146 |
| 4 | Gemma 4 26B A4B (IT) | 35.34 | #299 |
| 5 | Qwen 3.5 9B | 32.75 | #395 |
| 6 | EXAONE 4.5 33B | 25.86 | #402 |
| 7 | gemma-4-E4B-it | 18.62 | #577 |
| 8 | Qwen 3.5 4B | 17.58 | #516 |
| 9 | Ministral-3-8B-Instruct-2512 | 12.99 | #664 |
No result here: #3 Claude Opus 5.5, #5 GPT-6 Astra, #6 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=kopa-bench-pass-1 · How It Works · Data refreshed daily, snapshot 2026-11-07.