ISO-Bench (SGLang) - Hard Success: leaderboard
Metric: Hard success rate (%) on the 15 SGLang inference-optimization tasks: the patch beats or matches the human fix's measured speedup, whatever code it changes; each agent gets the repository at the commit before an expert's merged performance pull request, the benchmark showing the bottleneck and a 120-minute budget, and must write an optimization patch that keeps outputs identical; the patch's latency (TTFT) or throughput change is measured against the human fix (within 5 percent counts as similar); higher is better. Source: arxiv.org. Saturation forecast: Around January 2027. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5 (High) | 80 | #91 (GPT-5) |
| 2 | Claude Sonnet 4.5 | 46.7 | #138 |
No result here: #3 Claude Opus 5.5, #5 GPT-6 Astra, #8 Claude Fable 5.1.
Interactive version: theaggregate.ai/benchmark?slug=iso-bench-sglang-hard-success · How It Works · Data refreshed daily, snapshot 2026-10-11.