KnowU-Bench: leaderboard

Personalized and proactive mobile agents on KnowU-Bench (2026): 192 tasks in a live Android emulator (42 general, 86 personalized, 64 proactive), where the agent sees only the user behavior logs with irrelevant history mixed in, must infer or ask an LLM user simulator for preferences, and must decide when to act, seek consent or stay silent. Overall success rate in percent; higher is better.

Metric: Overall success rate (%) over all 192 tasks (42 general, 86 personalized, 64 proactive) in the reproducible KnowU-Bench Android emulator, noisy full-history memory setting (the complete user logs plus irrelevant history), gpt-4o as the user simulator grounded in a hidden user profile, scored by rule-based checks fused with a rubric-conditioned LLM judge; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 11 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Claude Sonnet 4.660.4#179
2Seed 2.0 Pro51.6#226
3Gemini 3.1 Pro (Preview)44.3#102
4Qwen 3.5 397B A17B37.5#270
5Qwen 3.5 9B33.3#838
6Qwen 3 VL 32B29.2#624
7Qwen 3.5 122B A10B27.1#350
8MAI-UI-8B26
9UI-Venus-1.5-8B26
10GUI-OWL-1.5-8B22.4
11Qwen 3 VL 8B21.9#927

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=knowu-bench · How It Works · Data refreshed daily, snapshot 2026-11-22.