SWE-fficiency - OpenHands v0.51.1: leaderboard
An OpenHands agent gets a full Python repository (numpy, pandas, scipy and six others) and a slow workload and must make the workload faster while the repository's own tests still pass; each model ran all 498 tasks once. The score is the speedup ratio: the agent's speedup divided by the expert patch's speedup, harmonic mean over tasks, with failed or empty patches counted at a floor (1.0 = expert parity, no upper bound); unlike SWE-bench it grades how much faster the code runs, not whether an issue is fixed. The authors ran these models in OpenHands v0.51.1 in 2025.
Metric: Speedup ratio (x expert). Source: raw.githubusercontent.com. Saturation forecast: Around January 2027. 7 models tracked. Updated 17 Nov 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5 | 0.1571 | #177 |
| 2 | Claude Sonnet 4.5 | 0.1162 | #258 |
| 3 | Claude Opus 4.1 | 0.0984 | #279 |
| 4 | Qwen 3 Coder Plus (2025-09-23) | 0.0676 | #708 |
| 5 | Kimi K2 0905 | 0.0537 | #564 |
| 6 | GLM-4.6 | 0.042 | #496 |
| 7 | GPT-5 Mini | 0.0389 | #359 |
No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.
Interactive version: theaggregate.ai/benchmark?slug=swe-fficiency-openhands-v0-51-1 · How It Works · Data refreshed daily, snapshot 2026-11-17.