ViBench 2.0 - Public8 Plan Pass: leaderboard
Long-horizon greenfield software building with sequential features and browser-graded test plans. Browser test-plan pass rate (%). ViBench2 native public OpenHands-SDK harness, medium reasoning, temperature1.0, max300steps and2hours per stage, summarization past200k tokens; provider default output/context limits and recommended open-weight top-p. Independent cohort ran2.0.1.beta, historical-GPT cohort2.0.5 with matching task/plan hashes. Four independent builds per product. Pass@1 requires every browser-graded plan to pass; Plan Pass is the mean fraction of plans passed. Grader/seeding Opus5.5 medium.
Metric: Browser test-plan pass rate (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-6 Astra (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 97.2 | |
| 2 | GPT-6.1 Sol (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 94.9 | |
| 3 | Claude Opus 5.5 (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 91.6 | |
| 4 | Claude Sonnet 5.5 (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 78.7 | |
| 5 | DeepSeek V4.1 Flash (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 72.1 | |
| 6 | GLM-5.3 (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 63.4 | |
| 7 | Kimi K3 (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 53.8 | |
| 8 | Grok 4.7 (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 41.1 | |
| 9 | Gemini 3.8 Flash (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 28.2 |
Interactive version: theaggregate.ai/benchmark?slug=vibench-2-0-public8-plan-pass · How It Works · Data refreshed daily, snapshot 2026-11-06.