LiveClawBench - Pass^3: leaderboard

Agent consistency on LiveClawBench (2026), personal-assistant agent tasks in reproducible full-stack mock services (stateful web applications, browser, APIs and shell) with implicit goals and runtime perturbations, three runs per case scored by case verifiers and a rubric judge. Share of the 134 cases in which all three runs earn a reward above 0.8, in percent; higher is better.

Metric: Pass^3 (x100): share of the 134 cases where all three runs have reward above 0.8, OpenClaw-style personal-assistant tasks run in reproducible full-stack mock services (22 stateful web applications, browser, APIs and shell), three runs per case, rewards from case verifiers and a DeepSeek-V3.2 rubric judge for open-ended cases; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 17 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1GPT-5.553.7#54
2Kimi K2.7 Code51.5#203
3GLM-5.149.3#198
4MiniMax-M2.747#539
5Qwen 3.6 Plus46.3#238
6GLM-5.246.3#150
7Claude Opus 4.845.5#65
8MiMo-V2.5-Pro43.3#284
9Qwen 3.6 Flash43.3#448
10MiniMax-M342.5#280
11Kimi K2.639.1#192
12DeepSeek V4 Pro32.8#196
13DeepSeek V4 Flash29.9#267
14Qwen 3.6 27B29.9#351
15Qwen 3.5 Flash23.1#441

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=liveclawbench-pass-pow-3 · How It Works · Data refreshed daily, snapshot 2026-11-21.