MLS-Bench (Vanilla): leaderboard
Metric: Mean of the 12 area scores, each the mean normalized task score (0-100): each metric is anchored so the worst reproduced human baseline scores 0 and the best 50 (a metric's theoretical optimum maps to 100), metrics are averaged with human weights within a setting and by geometric mean across at least three generalization settings, over the 140 method-invention tasks of MLS-Bench (the full set, not MLS-Bench-Lite); Vanilla protocol: the score of the model's first tested proposal (tools: edit, test, submit, undo); high reasoning effort with a 10,000-token thinking budget, web search disabled, fixed seeds; higher is better. Source: arxiv.org. Saturation forecast: Around 2029. 5 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | Claude Opus 4.6 (Thinking) | 28.1 |
| 2 | Gemini 3.1 Pro (Preview) (High) | 26.5 |
| 3 | GPT-5.4 (High) | 19.5 |
| 4 | DeepSeek V3.2 (Thinking) | 18.1 |
| 5 | Qwen 3.6 Plus (Thinking) | 16.5 |
Interactive version: theaggregate.ai/benchmark?slug=mls-bench-vanilla · How It Works · Data refreshed daily, snapshot 2026-10-07.