SWE Refactor Bench: leaderboard
A whole-repository stack migration held open for a hundred-odd steps: change the framework the codebase is built on and leave it working. Every model is run at each of its harness's effort tiers, and the best any of them manages is 47 out of 100 — the behavioural test suite mostly passes, the migration mostly does not.
Metric: Score (out of 100). Source: arxiv.org. Saturation forecast: Around October 2027. 26 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 5 (xHigh) | 47 | #12 (Claude Opus 5) |
| 2 | Claude Opus 5 (High) | 34.5 | #12 (Claude Opus 5) |
| 3 | Claude Opus 5 (Max) | 31 | #12 (Claude Opus 5) |
| 4 | GPT-5.6 Sol (Max) | 28.5 | #19 (GPT-5.6 Sol) |
| 5 | Claude Opus 5 (Medium) | 28.5 | #12 (Claude Opus 5) |
| 6 | Claude Opus 5 (Low) | 20.5 | #12 (Claude Opus 5) |
| 7 | Kimi K3 (Max) | 19.5 | #30 (Kimi K3) |
| 8 | GPT-5.6 Sol (High) | 19 | #19 (GPT-5.6 Sol) |
| 9 | Claude Sonnet 5 (Medium) | 15 | #57 (Claude Sonnet 5) |
| 10 | GPT-5.6 Luna (Max) | 10.5 | #68 (GPT-5.6 Luna) |
| 11 | Qwen 3.8 Max (Max) | 10 | #35 (Qwen 3.8 Max) |
| 12 | GPT-5.6 Sol (xHigh) | 9.5 | #19 (GPT-5.6 Sol) |
| 13 | Claude Sonnet 5 (xHigh) | 9 | #57 (Claude Sonnet 5) |
| 14 | Claude Sonnet 5 (Max) | 8.5 | #57 (Claude Sonnet 5) |
| 15 | DeepSeek V4 Flash (Max) | 7 | #137 (DeepSeek V4 Flash) |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=swe-refactor-bench · How It Works · Data refreshed daily, snapshot 2026-10-20.