KC-Bench - Personal Assistant: leaderboard
The multi-source-conflict domain of KC-Bench (Lyu et al., 2026): 74 personal-assistant tasks holding expired and active contact records for one person, which the agent must reconcile before communicating. One trial per task under a user simulator and stateful tools; a task succeeds only if the agent detects the conflict without exposing or changing protected state, as verified by human annotators; percent, higher is better.
Metric: Success rate (%; multi-source conflict: 74 personal-assistant tasks with expired and active contact records for one entity, which the agent must reconcile before communicating; one trial per task, greedy decoding, at most 30 interaction steps, LLM user simulator, stateful tools, success only if the conflict is detected and no protected state is exposed or modified; human-verified label (automatic assertion verdicts adjudicated by annotators)). Source: arxiv.org. Saturation forecast: Around February 2027. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3 Flash (Preview) | 79.73 | #141 |
| 2 | DeepSeek V4 Flash | 69.86 | #254 |
| 3 | MiniMax-M3 | 67.12 | #265 |
| 4 | GLM-5.2 | 65.75 | #137 |
| 5 | GPT-5.1 | 62.16 | #225 |
| 6 | Claude Haiku 4.5 (20251001) | 56.76 | #529 |
| 7 | Qwen 3.5 35B A3B | 37.83 | #480 |
| 8 | GLM 4.5 Air | 33.78 | #863 |
| 9 | GPT-OSS-120B | 22.97 | #699 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=kc-bench-personal-assistant · How It Works · Data refreshed daily, snapshot 2026-11-12.