ViBench 2.0 - All17 Plan Pass: leaderboard
Long-horizon greenfield software building with sequential features and browser-graded test plans. Browser test-plan pass rate (%). ViBench2 native public OpenHands-SDK harness, medium reasoning, temperature1.0, max300steps and2hours per stage, summarization past200k tokens; provider default output/context limits and recommended open-weight top-p. Independent cohort ran2.0.1.beta, historical-GPT cohort2.0.5 with matching task/plan hashes. Four independent builds per product. Pass@1 requires every browser-graded plan to pass; Plan Pass is the mean fraction of plans passed. Grader/seeding Opus5.5 medium.
Metric: Browser test-plan pass rate (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-6 Astra (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 98 | |
| 2 | GPT-6.1 Sol (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 96.9 | |
| 3 | Claude Opus 5.5 (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 93.7 | |
| 4 | Claude Sonnet 5.5 (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 79.2 | |
| 5 | DeepSeek V4.1 Flash (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 77.4 | |
| 6 | GLM-5.3 (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 66 | |
| 7 | Kimi K3 (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 56.7 | |
| 8 | Grok 4.7 (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 52.9 | |
| 9 | Gemini 3.8 Flash (Benchmark Sweep Run: source=ViBench2; OpenHandsSDK; medium; temperature1.0; cohort=independent17) | 38.1 |
Interactive version: theaggregate.ai/benchmark?slug=vibench-2-0-all17-plan-pass · How It Works · Data refreshed daily, snapshot 2026-11-06.