RoadmapBench - Web and RPC: leaderboard

RoadmapBench (2026) tests coding agents on 115 long-horizon tasks built from real open-source version upgrades in 17 repositories and five languages: from the earlier release and a roadmap of subtasks, the agent must implement the next release, with each subtask checked by its own tests. This board is the percentage of the 17 web and RPC tasks the agent fully resolves, every subtask test suite passing; higher is better.

Metric: Resolved rate (%) on the 17 web and RPC tasks (every subtask test suite passes), 115 long-horizon tasks implementing a real release upgrade from a roadmap of subtasks, one trial per model, 2-hour wall-clock budget, extended thinking on; each row names its agent scaffold (OpenHands, the primary scaffold, or Terminus 2, the ablation); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 13 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Claude Opus 4.7xHigh41.2#85
2Claude Opus 4.6Thinking29.4#113
3Gemini 3.1 Pro (Preview)High23.5#99
4GPT-5.4High17.6#135
5DeepSeek V4 ProHigh17.6#196
6Qwen 3.5 397B A17B11.8#264
7GLM-5.111.8#198
8MiMo-V2.5-Pro11.8#284
9Kimi K2.55.9#266
10Kimi K2.65.9#192
11Qwen 3.6 Plus5.9#238
12Seed 2.0 ProHigh5.9#222
13MiniMax-M2.70#539

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=roadmapbench-web-and-rpc · How It Works · Data refreshed daily, snapshot 2026-11-21.