SWE-Perf: leaderboard

Given the files and target functions of 140 real performance-improving pull requests from 9 Python repositories, each model writes one patch to make the code faster; the score is the mean, over the 140 instances, of the statistically significant minimum runtime gain on the related tests, in percent, with patches that fail to apply or break tests counting zero (the human expert patches reach 10.85%). Unlike SWE-bench it scores speedups of already-working code, not bug fixes.

Metric: Performance gain (%). Source: arxiv.org. Saturation forecast: Around 2030. 8 models tracked. Updated 17 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1Claude Sonnet 4Thinking1.76#411
2Gemini 2.5 Pro (Preview 05-06)1.48#271
3O3 (2025-04-16)1.37#229
4Claude Opus 4Thinking1.28#321
5Claude 3.7 SonnetThinking1.24#476
6DeepSeek R1 05280.9#431
7Qwen 3 235B A22B0.68#631
8GPT-4o (2024-11-20)0.6#864

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=swe-perf · How It Works · Data refreshed daily, snapshot 2026-11-17.