ViBench 2.0 - Public8 Pass@1: leaderboard
Long-horizon greenfield software building with sequential features and browser-graded test plans. Fully working build rate (%). ViBench2 native public OpenHands-SDK harness, medium reasoning, temperature1.0, max300steps and2hours per stage, summarization past200k tokens; provider default output/context limits and recommended open-weight top-p. Independent cohort ran2.0.1.beta, historical-GPT cohort2.0.5 with matching task/plan hashes. Four independent builds per product. Pass@1 requires every browser-graded plan to pass; Plan Pass is the mean fraction of plans passed. Grader/seeding Opus5.5 medium.
Metric: Fully working build rate (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-6.1 Sol (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 53.1 | |
| 2 | GPT-6 Astra (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 53.1 | |
| 3 | Claude Opus 5.5 (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 37.5 | |
| 4 | Claude Sonnet 5.5 (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 12.5 | |
| 5 | Grok 4.7 (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 3.1 | |
| 6 | DeepSeek V4.1 Flash (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 0 | |
| 7 | GLM-5.3 (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 0 | |
| 8 | Kimi K3 (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 0 | |
| 9 | Gemini 3.8 Flash (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 0 |
Interactive version: theaggregate.ai/benchmark?slug=vibench-2-0-public8-pass-1 · How It Works · Data refreshed daily, snapshot 2026-11-06.