SPU - Public - Raw: leaderboard
SPU - Public - Raw — SPU v1: rank wet-lab experiment outcomes; per-group Spearman correlation, combined with fixed source group weights within tasks (equal in97/100public tasks, unequal in3). 100 public/16 private tasks; one official run per configuration/task with one infrastructure retry. 8-hour budget,8CPU cores,16GiB RAM,H20GPU; results/reconstruction prohibited. GPT-6 Astra and Sol use Codex0.156.1; other models use Claude Code2.1.260 at max effort. Task-adjusted public score is the source additive task/configuration model, not a new local fit. Raw means use only valid or unambiguously recovered rankings; refusals/missing files excluded, not zero. Exact row counts and CIs remain provenance: public scored coverage30–100/100; private2–16/16. Private mean is unadjusted; it is not a matched-cohort comparison. The default-grader -1 convention is a separate diagnostic. Metric: Mean Spearman correlation.
Metric: Mean Spearman correlation. Source: arxiv.org. Saturation forecast: Around April 2028. 12 models tracked. Updated 15 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Max | 0.491 | #5 |
| 2 | Claude Fable 5.1 | Max | 0.453 | #15 |
| 3 | GPT-6 Astra | Medium | 0.431 | #6 |
| 4 | Claude Opus 5 | Max | 0.422 | #20 |
| 5 | Claude Sonnet 5.5 | Max | 0.388 | #19 |
| 6 | Doubao-Seed-Evolving-0909 | Max | 0.315 | |
| 7 | GPT-6.1 Sol | Max | 0.308 | #10 |
| 8 | Kimi K3.1 Pro | Max | 0.287 | #29 |
| 9 | DeepSeek V4.1 Flash | Max | 0.254 | #62 |
| 10 | Hy4 preview | Max | 0.245 | #71 |
| 11 | GLM-5.3 Flash | Max | 0.242 | #125 |
| 12 | Qwen 3.8 Max (0902) | Max | 0.224 | #49 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon.
Interactive version: theaggregate.ai/benchmark?slug=spu-public-raw · How It Works · Data refreshed daily, snapshot 2026-11-16.