LoopsBench - Vendor Loops - Test Pass Rate: leaderboard
LoopsBench (arXiv 2608.00267v2, Table 2 within-family panel): 12 models on 112 dependency-structured long-horizon coding tasks, each run in its vendor's loop (GPT in Codex, Claude in Claude Code, Qwen in Qwen Code). Mean fraction of released tests passing at task end after outer-loop continuation; percent, higher is better.
Metric: Released-test pass rate (%; with outer-loop continuation). Source: arxiv.org. Saturation forecast: Around February 2027. 12 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.7 | 53.05 | #51 |
| 2 | GPT-5.5 | 50.84 | #33 |
| 3 | Claude Opus 4.5 | 48.21 | #87 |
| 4 | Qwen 3.6 Plus | 45.1 | #124 |
| 5 | GPT-5.2 | 43.9 | #115 |
| 6 | Claude Sonnet 4.5 | 42.58 | #148 |
| 7 | Qwen 3.5 Plus | 41.17 | #133 |
| 8 | GPT-5 | 36.86 | #105 |
| 9 | Claude Sonnet 4 | 35.54 | #207 |
| 10 | GPT-4o | 28.3 | #357 |
| 11 | Qwen 3 Max | 23.31 | #213 |
| 12 | Qwen 2.5 72B | 13.88 | #512 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=loopsbench-vendor-loops-test-pass-rate · How It Works · Data refreshed daily, snapshot 2026-11-05.