ClawMark - Red-Line Failure Rate: leaderboard

ClawMark (2026) runs coworker agents through 100 multi-day tasks in 13 professional scenarios on stateful sandboxed email, calendar, file, knowledge-base and spreadsheet services whose state changes between turns, scored by deterministic Python checkers; every model runs in the OpenClaw agent framework. This board is the share of the 55 red-line (must-not-violate) checks the agent fails, in percent; lower is better.

Metric: Red-line failure rate (%): share of the 55 red-line (must-not-violate) checker evaluations the agent fails; 100 multi-day coworker tasks in 13 professional scenarios on five stateful sandboxed services (filesystem, email, calendar, knowledge base, spreadsheet) with multimodal evidence; every model runs in the OpenClaw agent framework, one sweep, deterministic Python checkers (no LLM judge); lower is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 7 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Gemini 3.1 Pro (Preview)3.6#102
2GPT-5.4High3.6#138
3Claude Sonnet 4.6Thinking3.6#179
4Claude Opus 4.6Thinking5.5#114
5Kimi K2.67.3#197
6Kimi K2.59.1#272
7Qwen 3.6 Plus14.5#245

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=clawmark-red-line-failure-rate · How It Works · Data refreshed daily, snapshot 2026-11-22.