OWE-Bench - INT4 Decode Attention: leaderboard

OWE-Bench - INT4 Decode Attention — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.

Metric: Mean normalized held-out task score (fraction, 0–1). Source: openworldengineer.mercor.com. Saturation forecast: Around June 2027. 14 models tracked. Updated 15 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1Claude Opus 5.5Max0.627#5
2Claude Fable 5.2Max0.5961#1
3Claude Sonnet 5.5Max0.566#19
4GPT-6 AstraMax0.4705#6
5GPT-6.1 SolMax0.4668#10
6Gemini 4 CarbonHigh0.4367#2
7Claude Fable 5.1Max0.4331#15
8Gemini 4 ArgonHigh0.3757#12
9GLM-5.3Max0.2128#68
10DeepSeek V4.1 FlashMax0.1827#62
11Grok 4.7xHigh0.1601#47
12Muse WonderMax0.159#28
13Beam 1.0Max0.03465#206
14InklingMax0.002125#281

Interactive version: theaggregate.ai/benchmark?slug=owe-bench-int4-decode-attention · How It Works · Data refreshed daily, snapshot 2026-11-16.