Singularity-Bench - RL SWE Agent: leaderboard

Singularity-Bench - RL SWE Agent — Singularity-Bench v1 Table5; six engineering tasks with native model/harness pairs. Pretraining,midtraining,RL and engine-building each12hours on4GB300; coding-harness6hours on1GB300; serverless-cluster4hours on16GB300. Held-out downstream task performance, not builder output speed or cost. Stars mean beating the human baseline; exact stars and native cost cells remain in row provenance. Section6.3binds the Opus5.5andSol inference-engine cells to max effort; unreported settings are not inferred for other cells. No lowest-cost or best-harness pooling. Crosses denote failed/rejected submissions; their individual raw metrics remain absent rather than invented numeric values. AppendixA.1 assigns those failures zero contribution to the published Index, which is retained natively. Human baseline is an engineering reference, not a fitted population median. Separate harnesses retain their native labels and documented source controls. Metric: Mean resolved fraction.

Metric: Mean resolved fraction. Source: arxiv.org. Saturation forecast: Around October 2028. 7 models tracked. Updated 15 Nov 2026.

Top models

#ModelScoreOverall rank
1Claude Opus 5.50.533#5
2GPT-6.1 Sol0.53#10
3Claude Opus 50.517#20
4DeepSeek V4.1 Flash0.437#62
5Kimi K30.422#56
6GLM-5.30.41#68
7MiniMax-M30.392#272

No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #6 GPT-6 Astra.

Interactive version: theaggregate.ai/benchmark?slug=singularity-bench-rl-swe-agent · How It Works · Data refreshed daily, snapshot 2026-11-16.