SmellBench Refactoring (Qwen Code) - Localization Accuracy: leaderboard
SmellBench (2026, code refactoring) injects code smells into clean code from seven real Python repositories and asks a code agent to remove them; it is unrelated to the SmellBench board on architectural smell repair. This board is the percentage of cases in which the agent edits the class or method that holds the injected smell, over 294 cases with the Qwen Code agent harness; higher is better.
Metric: Localization accuracy (%): the edited class or method matches the ground-truth refactoring target; 294 injected-smell refactoring cases (7 smell types, 3 difficulty levels, guided and targeted instructions) in 7 real Python repositories, run once in the Harbor Docker framework with a 20-minute budget, Qwen Code agent harness; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4.5 | 88.78 | #250 |
| 2 | GPT-5 Mini | 74.83 | #348 |
| 3 | Qwen 3 Coder 30B A3B Instruct | 67.01 | #972 |
| 4 | Qwen 3 Coder 480B A35B Instruct | 65.31 | #643 |
| 5 | Gemini 2.5 Flash | 63.95 | #453 |
| 6 | DeepSeek V3.2 | 49.66 | #406 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=smellbench-refactoring-qwen-code-localization-accuracy · How It Works · Data refreshed daily, snapshot 2026-11-13.