SWE-Race - Public: leaderboard
SWE-Race public subset: repair real Python concurrency and async-lifecycle bugs using the BenchGen coding agent at medium effort and a 100-step offline budget. Score is mean per-task pass@1 over repeated attempts on shipped task versions; all hidden fail-to-pass tests must pass without regressions. Native task and attempt counts differ by model.
Metric: Mean per-task pass@1 (%). Source: labs.evaligo.com. Saturation forecast: Rough model projection: around 2026. 3 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.6 Luna (Medium) (SWE-Race Run: v0.1 BenchGen) | 82.9 | |
| 2 | GLM-5.3 Flash (Medium) (SWE-Race Run: v0.1 BenchGen) | 80.2 | |
| 3 | DeepSeek V4 Flash (Medium) (SWE-Race Run: v0.1 BenchGen) | 69.2 |
Interactive version: theaggregate.ai/benchmark?slug=swe-race-public · How It Works · Data refreshed daily, snapshot 2026-11-05.