LoopsBench - Vendor Loops - Resolve Rate: leaderboard
LoopsBench (arXiv 2608.00267v2, Table 2 within-family panel): 12 models on 112 dependency-structured long-horizon coding tasks, each run in its vendor's loop (GPT in Codex, Claude in Claude Code, Qwen in Qwen Code). Share of tasks where every released test, regression obligations included, passes at task end after outer-loop continuation rounds on the preserved workspace; percent, higher is better.
Metric: Resolve rate (%; with outer-loop continuation). Source: arxiv.org. Saturation forecast: Around June 2027. 12 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.7 | 25 | #51 |
| 2 | GPT-5.5 | 21.43 | #33 |
| 3 | Claude Opus 4.5 | 17.86 | #87 |
| 4 | GPT-5.2 | 13.39 | #115 |
| 5 | Claude Sonnet 4.5 | 12.5 | #148 |
| 6 | Qwen 3.6 Plus | 9.82 | #124 |
| 7 | GPT-5 | 8.04 | #105 |
| 8 | Qwen 3.5 Plus | 8.04 | #133 |
| 9 | Claude Sonnet 4 | 7.14 | #207 |
| 10 | GPT-4o | 3.57 | #357 |
| 11 | Qwen 3 Max | 1.79 | #213 |
| 12 | Qwen 2.5 72B | 0 | #512 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=loopsbench-vendor-loops-resolve-rate · How It Works · Data refreshed daily, snapshot 2026-11-05.