SmellBench Refactoring (OpenHands) - Localization Accuracy: leaderboard
SmellBench (2026, code refactoring) injects code smells into clean code from seven real Python repositories and asks a code agent to remove them; it is unrelated to the SmellBench board on architectural smell repair. This board is the percentage of cases in which the agent edits the class or method that holds the injected smell, over 294 cases with the OpenHands agent harness; higher is better.
Metric: Localization accuracy (%): the edited class or method matches the ground-truth refactoring target; 294 injected-smell refactoring cases (7 smell types, 3 difficulty levels, guided and targeted instructions) in 7 real Python repositories, run once in the Harbor Docker framework with a 20-minute budget, OpenHands agent harness; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4.5 | 84.01 | #250 |
| 2 | GPT-5 Mini | 77.21 | #348 |
| 3 | Qwen 3 Coder 480B A35B Instruct | 74.56 | #643 |
| 4 | DeepSeek V3.2 | 72.24 | #406 |
| 5 | Qwen 3 Coder 30B A3B Instruct | 70.67 | #972 |
| 6 | Gemini 2.5 Flash | 66.67 | #453 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=smellbench-refactoring-openhands-localization-accuracy · How It Works · Data refreshed daily, snapshot 2026-11-13.