SWE-fficiency - OpenHands v0.51.1: leaderboard

An OpenHands agent gets a full Python repository (numpy, pandas, scipy and six others) and a slow workload and must make the workload faster while the repository's own tests still pass; each model ran all 498 tasks once. The score is the speedup ratio: the agent's speedup divided by the expert patch's speedup, harmonic mean over tasks, with failed or empty patches counted at a floor (1.0 = expert parity, no upper bound); unlike SWE-bench it grades how much faster the code runs, not whether an issue is fixed. The authors ran these models in OpenHands v0.51.1 in 2025.

Metric: Speedup ratio (x expert). Source: raw.githubusercontent.com. Saturation forecast: Around January 2027. 7 models tracked. Updated 17 Nov 2026.

Top models

#ModelScoreOverall rank
1GPT-50.1571#177
2Claude Sonnet 4.50.1162#258
3Claude Opus 4.10.0984#279
4Qwen 3 Coder Plus (2025-09-23)0.0676#708
5Kimi K2 09050.0537#564
6GLM-4.60.042#496
7GPT-5 Mini0.0389#359

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=swe-fficiency-openhands-v0-51-1 · How It Works · Data refreshed daily, snapshot 2026-11-17.