SWT-Bench Verified: leaderboard
SWT-Bench (Mündler et al., 2024) asks a system to write tests that reproduce a user-reported GitHub issue: a test must fail on the original code and pass once the reference fix is applied. Verified uses 433 human-verified solvable issues matching SWE-bench Verified, and the score is the share of issues reproduced (%) by tests added to the repository's test suite (unit test mode); higher is better. Rows are agent or method systems with their models.
Metric: Success rate S (%, issues reproduced, unit test mode). Source: swtbench.com. Saturation forecast: Around February 2027. 19 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 5.5 | 87.8 | #5 |
| 2 | GPT-5 | 79.8 | #163 |
| 3 | Claude Sonnet 5.5 | 75.8 | #18 |
| 4 | GPT-5 Mini | 62.4 | #349 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #6 GPT-6 Astra.
Interactive version: theaggregate.ai/benchmark?slug=swt-bench-verified · How It Works · Data refreshed daily, snapshot 2026-11-12.