SPU - Private - Raw: leaderboard

SPU - Private - Raw — SPU v1: rank wet-lab experiment outcomes; per-group Spearman correlation, combined with fixed source group weights within tasks (equal in97/100public tasks, unequal in3). 100 public/16 private tasks; one official run per configuration/task with one infrastructure retry. 8-hour budget,8CPU cores,16GiB RAM,H20GPU; results/reconstruction prohibited. GPT-6 Astra and Sol use Codex0.156.1; other models use Claude Code2.1.260 at max effort. Task-adjusted public score is the source additive task/configuration model, not a new local fit. Raw means use only valid or unambiguously recovered rankings; refusals/missing files excluded, not zero. Exact row counts and CIs remain provenance: public scored coverage30–100/100; private2–16/16. Private mean is unadjusted; it is not a matched-cohort comparison. The default-grader -1 convention is a separate diagnostic. Metric: Mean Spearman correlation.

Metric: Mean Spearman correlation. Source: arxiv.org. Saturation forecast: Around 2029. 12 models tracked. Updated 15 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1Claude Opus 5.5Max0.42#5
2Claude Opus 5Max0.405#20
3Qwen 3.8 Max (0902)Max0.392#49
4Claude Sonnet 5.5Max0.391#19
5Hy4 previewMax0.36#71
6GPT-6.1 SolMax0.346#10
7DeepSeek V4.1 FlashMax0.324#62
8GPT-6 AstraMax0.322#6
9Doubao-Seed-Evolving-0909Max0.32
10Kimi K3.1 ProMax0.264#29
11Claude Fable 5.1Max0.214#15
12GLM-5.3 FlashMax0.206#125

No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=spu-private-raw · How It Works · Data refreshed daily, snapshot 2026-11-16.