SWT-Bench: leaderboard
SWT-Bench: Evaluates software-engineering agents on realistic issue resolution, repository navigation, testing, or maintenance workflows.
Metric: Success rate S (%, issues reproduced, unit test mode). Source: swtbench.com. 17 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Mistral Large 2 (Jul) | 16.3 | #484 |
| 2 | GPT-4 Preview (1106) | 15.9 | #418 |
| 3 | Claude 3.5 Sonnet | 12.3 | #350 |
| 4 | GPT-4o Mini (2024-07-18) | 9.8 | #558 |
| 5 | Claude 3 Haiku | 2.5 | #803 |
| 6 | Mixtral 8x22B | 0.7 | #766 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=swt-bench · How It Works · Data refreshed daily, snapshot 2026-10-20.