Singularity-Bench - RL SWE Agent: leaderboard
Singularity-Bench - RL SWE Agent — Singularity-Bench v1 Table5; six engineering tasks with native model/harness pairs. Pretraining,midtraining,RL and engine-building each12hours on4GB300; coding-harness6hours on1GB300; serverless-cluster4hours on16GB300. Held-out downstream task performance, not builder output speed or cost. Stars mean beating the human baseline; exact stars and native cost cells remain in row provenance. Section6.3binds the Opus5.5andSol inference-engine cells to max effort; unreported settings are not inferred for other cells. No lowest-cost or best-harness pooling. Crosses denote failed/rejected submissions; their individual raw metrics remain absent rather than invented numeric values. AppendixA.1 assigns those failures zero contribution to the published Index, which is retained natively. Human baseline is an engineering reference, not a fitted population median. Separate harnesses retain their native labels and documented source controls. Metric: Mean resolved fraction.
Metric: Mean resolved fraction. Source: arxiv.org. Saturation forecast: Around October 2028. 7 models tracked. Updated 15 Nov 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 5.5 | 0.533 | #5 |
| 2 | GPT-6.1 Sol | 0.53 | #10 |
| 3 | Claude Opus 5 | 0.517 | #20 |
| 4 | DeepSeek V4.1 Flash | 0.437 | #62 |
| 5 | Kimi K3 | 0.422 | #56 |
| 6 | GLM-5.3 | 0.41 | #68 |
| 7 | MiniMax-M3 | 0.392 | #272 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #6 GPT-6 Astra.
Interactive version: theaggregate.ai/benchmark?slug=singularity-bench-rl-swe-agent · How It Works · Data refreshed daily, snapshot 2026-11-16.