JamBench - Function Completion (Medium Projects): leaderboard
JamBench (2026) tests project-level game programming on the Godot engine with real Game Jam projects; outputs are checked deterministically in headless mode, and the board is the percentage of projects that compile and then run for 30 seconds without crashing. In this completion task on medium projects (4K to 15K lines), 30 to 50% of function bodies are cleared and must be rewritten, with the model writing the missing code directly; higher is better.
Metric: L3a runtime pass rate (%): share of generated Godot 4 projects that pass file-integrity and headless compilation checks and then run 30 seconds in headless mode without crashing; Task 2 code completion: 100 benchmark projects with medium code bases (4K to 15K lines), each manually verified playable; 30 to 50% of function bodies are cleared and must be rewritten; direct generation, the model writes every file in one pass; higher is better. Source: arxiv.org. Saturation forecast: Around May 2027. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.6 | 44 | #102 |
| 2 | Kimi K2.5 | 43 | #256 |
| 3 | Qwen 3.5 397B A17B | 41 | #246 |
| 4 | GPT-5.4 | 38 | #123 |
| 5 | GLM-5 | 37 | #260 |
| 6 | DeepSeek V4 Pro | 34 | #188 |
| 7 | Gemini 3.1 Pro (Preview) | 32 | #94 |
| 8 | Qwen 3.5 27B | 17 | #359 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=jambench-function-completion-medium-projects · How It Works · Data refreshed daily, snapshot 2026-11-13.