ControlHarbor (Claude Code, 1-hour session) - crowd-bottleneck-56: leaderboard
Models and harness: Claude Opus 5.5, Claude Fable 5.1, Claude Sonnet 5.5 via Claude Code at the highest reasoning effort (level name not printed), launched by Harbor; versions of Claude Code and Harbor not stated (§3; Appendix D). Session: One container, one instruction, 60 minutes from the first command, one graded Python file; no network; 2 cores, 2 GB; agent setup allowed 5x Harbor default; no neural networks or gradient-based learning in the artefact (§2; §3; Appendix A.2; Appendix D). Grading: Separate container, 15-minute verifier timeout, held-out seeds (40 on locomotion), chance-constrained gate with epsilon 0.1 per plant model, return bound rho 0.9 and certificate where declared; reward 1 or 0 (§2.1; Appendix A). Cohorts: Opus and Fable: all 44 tasks (132 and 131 sessions); Sonnet: 22 MuJoCo tasks (67 sessions); per-cell session counts 2-4 as printed (§3; A2.T3; A3.T5). Task variants: -handoff tasks start from another model's passing controller of the tier below; -notune tasks remove the tuner; soft-arm tasks (A6.T7) run after the main sweep under the same protocol (§5; Appendix F). This board: Sessions per model as printed (usually 3; 2 or 4 where printed); Count of passing sessions over independent sessions; no seeds averaged across models; "Three sessions per cell give counts, not rates" (§6). ControlHarbor v1 task "crowd-bottleneck-56" (repository https://github.com/jq-wei/hl-control; commit not stated); Sonnet 5.5 was not run on non-MuJoCo tasks. Metric: Session pass rate: sessions whose single submitted controller file passes the task gate (every required rung on at least 0.9 of held-out seeds on the nominal and every perturbed plant, plus return bound/certificate where declared) divided by sessions run Higher is better.
Metric: Session pass rate: sessions whose single submitted controller file passes the task gate (every required rung on at least 0.9 of held-out seeds on the nominal and every perturbed plant, plus return bound/certificate where declared) divided by sessions run. Source: arxiv.org. Saturation forecast: Around February 2027. 2 models tracked. Updated 14 Nov 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 5.5 | 1 | #5 |
| 2 | Claude Fable 5.1 | 1 | #15 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #6 GPT-6 Astra.
Interactive version: theaggregate.ai/benchmark?slug=controlharbor-claude-code-1-hour-session-crowd-bottleneck-56 · How It Works · Data refreshed daily, snapshot 2026-11-16.