KnowU-Bench: leaderboard
Personalized and proactive mobile agents on KnowU-Bench (2026): 192 tasks in a live Android emulator (42 general, 86 personalized, 64 proactive), where the agent sees only the user behavior logs with irrelevant history mixed in, must infer or ask an LLM user simulator for preferences, and must decide when to act, seek consent or stay silent. Overall success rate in percent; higher is better.
Metric: Overall success rate (%) over all 192 tasks (42 general, 86 personalized, 64 proactive) in the reproducible KnowU-Bench Android emulator, noisy full-history memory setting (the complete user logs plus irrelevant history), gpt-4o as the user simulator grounded in a hidden user profile, scored by rule-based checks fused with a rubric-conditioned LLM judge; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 11 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4.6 | 60.4 | #179 |
| 2 | Seed 2.0 Pro | 51.6 | #226 |
| 3 | Gemini 3.1 Pro (Preview) | 44.3 | #102 |
| 4 | Qwen 3.5 397B A17B | 37.5 | #270 |
| 5 | Qwen 3.5 9B | 33.3 | #838 |
| 6 | Qwen 3 VL 32B | 29.2 | #624 |
| 7 | Qwen 3.5 122B A10B | 27.1 | #350 |
| 8 | MAI-UI-8B | 26 | |
| 9 | UI-Venus-1.5-8B | 26 | |
| 10 | GUI-OWL-1.5-8B | 22.4 | |
| 11 | Qwen 3 VL 8B | 21.9 | #927 |
No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=knowu-bench · How It Works · Data refreshed daily, snapshot 2026-11-22.