ProgramDistill: leaderboard
Microsoft Research's benchmark of reference-guided web development: a coding agent gets a web application with one behavior masked out and a running copy of the original, which it can use but whose source it cannot read, and must restore the behavior in code. Each fix is checked by replaying the recorded browser traces. The board is the public 300-task subset across 26 applications, at restoration depths 1 to 8, all run in one modified R2E-Gym harness. Binary score: the share of tasks whose every replayed behavior passes.
Metric: Binary score (%). Source: microsoft.github.io. Saturation forecast: Around February 2027. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-6 Astra | 84.33 | #6 |
| 2 | Claude Opus 5 | 68.67 | #20 |
| 3 | GPT-5.6 Sol | 60.67 | #29 |
| 4 | Grok 4.6 | 48.33 | #44 |
| 5 | Claude Sonnet 5 | 47.33 | #108 |
| 6 | GPT-5.3 Codex | 45.67 | #114 |
| 7 | Gemini 3.7 Flash | 45.33 | #32 |
| 8 | Gemini 3.6 Flash | 29.67 | #72 |
| 9 | Gemini 3.1 Pro (Preview) | 22.33 | #91 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=programdistill · How It Works · Data refreshed daily, snapshot 2026-11-12.