Singularity-Bench - Midtraining: leaderboard

Singularity-Bench - Midtraining — Singularity-Bench v1 Table5; six engineering tasks with native model/harness pairs. Pretraining,midtraining,RL and engine-building each12hours on4GB300; coding-harness6hours on1GB300; serverless-cluster4hours on16GB300. Held-out downstream task performance, not builder output speed or cost. Stars mean beating the human baseline; exact stars and native cost cells remain in row provenance. Section6.3binds the Opus5.5andSol inference-engine cells to max effort; unreported settings are not inferred for other cells. No lowest-cost or best-harness pooling. Crosses denote failed/rejected submissions; their individual raw metrics remain absent rather than invented numeric values. AppendixA.1 assigns those failures zero contribution to the published Index, which is retained natively. Human baseline is an engineering reference, not a fitted population median. Separate harnesses retain their native labels and documented source controls. Metric: Mean post-RL accuracy (fraction, 0–1).

Metric: Mean post-RL accuracy (fraction, 0–1). Source: arxiv.org. Saturation forecast: Around June 2028. 7 models tracked. Updated 15 Nov 2026.

Top models

#ModelScoreOverall rank
1Claude Opus 5.50.319#5
2Claude Opus 50.201#20
3GPT-6.1 Sol0.178#10
4DeepSeek V4.1 Flash0.134#62
5GLM-5.30.099#68
6MiniMax-M30.091#272
7Kimi K30.051#56

No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #6 GPT-6 Astra.

Interactive version: theaggregate.ai/benchmark?slug=singularity-bench-midtraining · How It Works · Data refreshed daily, snapshot 2026-11-16.