SWE-Perf: leaderboard
Given the files and target functions of 140 real performance-improving pull requests from 9 Python repositories, each model writes one patch to make the code faster; the score is the mean, over the 140 instances, of the statistically significant minimum runtime gain on the related tests, in percent, with patches that fail to apply or break tests counting zero (the human expert patches reach 10.85%). Unlike SWE-bench it scores speedups of already-working code, not bug fixes.
Metric: Performance gain (%). Source: arxiv.org. Saturation forecast: Around 2030. 8 models tracked. Updated 17 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Claude Sonnet 4 | Thinking | 1.76 | #411 |
| 2 | Gemini 2.5 Pro (Preview 05-06) | 1.48 | #271 | |
| 3 | O3 (2025-04-16) | 1.37 | #229 | |
| 4 | Claude Opus 4 | Thinking | 1.28 | #321 |
| 5 | Claude 3.7 Sonnet | Thinking | 1.24 | #476 |
| 6 | DeepSeek R1 0528 | 0.9 | #431 | |
| 7 | Qwen 3 235B A22B | 0.68 | #631 | |
| 8 | GPT-4o (2024-11-20) | 0.6 | #864 |
No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.
Interactive version: theaggregate.ai/benchmark?slug=swe-perf · How It Works · Data refreshed daily, snapshot 2026-11-17.