ScratchWorld - Create: leaderboard
Create tasks from ScratchWorld (Zhang et al., 2026): build a working Scratch project from a natural-language specification, checked by unit tests run in the Scratch VM. Composite mode, where the agent reads the program as pseudocode and edits it through semantic commands such as adding or connecting blocks instead of the GUI. The score is the percentage of the 20 tasks on which every unit test passes; higher is better.
Metric: Success Rate (%). Source: huggingface.co. Saturation forecast: Estimated already saturated. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 2.5 Pro | 90 | #152 |
| 2 | Claude Sonnet 4.5 | 90 | #142 |
| 3 | GPT-5 | 80 | #98 |
| 4 | DeepSeek V3.2 | 75 | #205 |
| 5 | Qwen 3 Coder 30B A3B Instruct | 10 | #415 |
| 6 | Qwen 2.5 Coder 7B Instruct | 0 | #951 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=scratchworld-create · How It Works · Data refreshed daily, snapshot 2026-10-20.