RoadmapBench - ML and Data: leaderboard
RoadmapBench (2026) tests coding agents on 115 long-horizon tasks built from real open-source version upgrades in 17 repositories and five languages: from the earlier release and a roadmap of subtasks, the agent must implement the next release, with each subtask checked by its own tests. This board is the percentage of the 36 machine learning and data tasks the agent fully resolves, every subtask test suite passing; higher is better.
Metric: Resolved rate (%) on the 36 machine learning and data tasks (every subtask test suite passes), 115 long-horizon tasks implementing a real release upgrade from a roadmap of subtasks, one trial per model, 2-hour wall-clock budget, extended thinking on; each row names its agent scaffold (OpenHands, the primary scaffold, or Terminus 2, the ablation); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 13 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Claude Opus 4.7 | xHigh | 30.6 | #85 |
| 2 | GPT-5.4 | High | 27.8 | #135 |
| 3 | Claude Opus 4.6 | Thinking | 25 | #113 |
| 4 | GLM-5.1 | 8.3 | #198 | |
| 5 | Gemini 3.1 Pro (Preview) | High | 8.3 | #99 |
| 6 | MiMo-V2.5-Pro | 8.3 | #284 | |
| 7 | DeepSeek V4 Pro | High | 8.3 | #196 |
| 8 | Kimi K2.6 | 5.6 | #192 | |
| 9 | Qwen 3.6 Plus | 5.6 | #238 | |
| 10 | MiniMax-M2.7 | 5.6 | #539 | |
| 11 | Kimi K2.5 | 0 | #266 | |
| 12 | Qwen 3.5 397B A17B | 0 | #264 | |
| 13 | Seed 2.0 Pro | High | 0 | #222 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=roadmapbench-ml-and-data · How It Works · Data refreshed daily, snapshot 2026-11-21.