KC-Bench - Personal Assistant: leaderboard
Metric: Success rate (%; multi-source conflict: 74 personal-assistant tasks with expired and active contact records for one entity, which the agent must reconcile before communicating; one trial per task, greedy decoding, at most 30 interaction steps, LLM user simulator, stateful tools, success only if the conflict is detected and no protected state is exposed or modified; human-verified label (automatic assertion verdicts adjudicated by annotators)). Source: arxiv.org. Saturation forecast: Around December 2026. 9 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | Gemini 3 Flash (Preview) | 79.73 |
| 2 | DeepSeek V4 Flash | 69.86 |
| 3 | MiniMax-M3 | 67.12 |
| 4 | GLM-5.2 | 65.75 |
| 5 | GPT-5.1 | 62.16 |
| 6 | Claude Haiku 4.5 (20251001) | 56.76 |
| 7 | Qwen 3.5 35B A3B | 37.83 |
| 8 | GLM 4.5 Air | 33.78 |
| 9 | GPT-OSS-120B | 22.97 |
Interactive version: theaggregate.ai/benchmark?slug=kc-bench-personal-assistant · How It Works · Data refreshed daily, snapshot 2026-09-26.