ScratchWorld - Debug (Primitive Mode): leaderboard
Debug tasks from ScratchWorld (Zhang et al., 2026): find and repair the logical errors in a broken Scratch project so that it behaves as specified, checked by unit tests run in the Scratch VM. Primitive mode, where the agent works from screenshots and must edit the blocks through low-level GUI actions such as click and drag-and-drop. The score is the percentage of the 20 tasks on which every unit test passes; higher is better.
Metric: Success Rate (%). Source: huggingface.co. Saturation forecast: Around February 2027. 11 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5 | 40 | #163 |
| 2 | Claude Sonnet 4.5 | 25 | #244 |
| 3 | Gemini 2.5 Pro | 20 | #262 |
| 4 | Qwen 3 VL 32B Instruct | 10 | #582 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=scratchworld-debug-primitive-mode · How It Works · Data refreshed daily, snapshot 2026-11-12.