KOPA-Bench - Pass@4: leaderboard
KOPA-Bench evaluates multi-step tool use over Korean public APIs on 145 tasks, with four rollouts per task. Pass@1 averages single-attempt success, pass@4 measures at least one success, and Action measures tool-call accuracy against golden actions. The paper's GRPO-tuned models remain distinct from their base checkpoints. This series reports KOPA-Bench - Pass@4 from 2609.05395v1, S3.T1: Pass@4 (%). Higher is better.
Metric: Pass@4 (%). Source: arxiv.org. Saturation forecast: Around February 2027. 11 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4.6 | 82.07 | #97 |
| 2 | Qwen 3.5 27B | 56.55 | #194 |
| 3 | Qwen 3.5 9B | 46.9 | #395 |
| 4 | GPT-5.1 | 46.9 | #146 |
| 5 | Gemma 4 26B A4B (IT) | 46.9 | #299 |
| 6 | EXAONE 4.5 33B | 38.62 | #402 |
| 7 | Qwen 3.5 4B | 31.03 | #516 |
| 8 | Ministral-3-8B-Instruct-2512 | 23.08 | #664 |
| 9 | gemma-4-E4B-it | 22.07 | #577 |
No result here: #3 Claude Opus 5.5, #5 GPT-6 Astra, #6 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=kopa-bench-pass-4 · How It Works · Data refreshed daily, snapshot 2026-11-07.