MLS-Bench (Vanilla) - Structured and Causal Reasoning: leaderboard

MLS-Bench (Vanilla) Structured and Causal Reasoning scores the first proposal each model tests on the 10 Structured and Causal Reasoning tasks of MLS-Bench (2026), where a model must invent a better ML method inside a frozen research codebase. Task scores are normalized so the worst reproduced human baseline is 0 and the best is 50, on a 0 to 100 scale where higher is better.

Metric: Mean normalized task score (0-100): each metric is anchored so the worst reproduced human baseline scores 0 and the best 50 (a metric's theoretical optimum maps to 100), metrics are averaged with human weights within a setting and by geometric mean across at least three generalization settings, over the 10 MLS-Bench tasks of the Structured and Causal Reasoning area; Vanilla protocol: the score of the model's first tested proposal (tools: edit, test, submit, undo); high reasoning effort with a 10,000-token thinking budget, web search disabled, fixed seeds; higher is better. Source: arxiv.org. Saturation forecast: Around 2028. 5 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Claude Opus 4.6Thinking26.6#113
2DeepSeek V3.2Thinking21.6#419
3GPT-5.4High15.8#135
4Gemini 3.1 Pro (Preview)High12.4#99
5Qwen 3.6 PlusThinking1.9#238

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=mls-bench-vanilla-structured-and-causal-reasoning · How It Works · Data refreshed daily, snapshot 2026-11-21.