ServerBench - Qwen3.5 9B 2xL40: leaderboard
Coding agents construct complete verified serving systems under frozen constraints. ServerBench v1 gives each coding agent one eight-hour development run in a network-disabled Harbor sandbox, followed by a 1,800-second replay on held-out seeds. All agents use maximum reasoning effort; GPT models use Codex and other models use Claude Code. The reported score belongs to the strongest verified final submission, allowing at most one rerun for a clearly anomalous failure. It measures latency-qualified token value per GPU dollar of the serving code the agent produced, under fixed checkpoint, hardware, latency targets and prices. Submission concurrency is retained. Native Not passed cells have no numeric score and are not converted to zero. Question access: Serving task code and development procedures are public; independent correctness-test details are withheld.
Metric: Latency-qualified dollars per GPU dollar. Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 7 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 5.5 (Benchmark Sweep Run: source=ServerBench; harness=Claude Code; maximum reasoning; serving=Qwen3.5 9B 2xL40; concurrency=16) | 4.4 | |
| 2 | Claude Sonnet 5.5 (Benchmark Sweep Run: source=ServerBench; harness=Claude Code; maximum reasoning; serving=Qwen3.5 9B 2xL40; concurrency=20) | 4.34 | |
| 3 | GPT 6 Astra (Benchmark Sweep Run: source=ServerBench; harness=Codex; maximum reasoning; serving=Qwen3.5 9B 2xL40; concurrency=12) | 3.18 | |
| 4 | GPT 6.1 Sol (Benchmark Sweep Run: source=ServerBench; harness=Codex; maximum reasoning; serving=Qwen3.5 9B 2xL40; concurrency=4) | 2.84 | |
| 5 | Claude Opus 5 (Benchmark Sweep Run: source=ServerBench; harness=Claude Code; maximum reasoning; serving=Qwen3.5 9B 2xL40; concurrency=1) | 1.16 | |
| 6 | GPT 6 Sol (Benchmark Sweep Run: source=ServerBench; harness=Codex; maximum reasoning; serving=Qwen3.5 9B 2xL40; concurrency=1) | 1.11 | |
| 7 | Kimi K3 (Benchmark Sweep Run: source=ServerBench; harness=Claude Code; maximum reasoning; serving=Qwen3.5 9B 2xL40; concurrency=1) | 0.93 |
Interactive version: theaggregate.ai/benchmark?slug=serverbench-qwen3-5-9b-2xl40 · How It Works · Data refreshed daily, snapshot 2026-11-06.