FullStackBench (2024-12 bilingual): leaderboard
FullStackBench December 2024 executable code-generation release: 3,374 problems split equally between English and Chinese, across 16 programming languages and 11 main application domains. Pass@1 is reproduced from the publisher's per-problem accepted outcomes; kept separate from language-specific runs.
Metric: Pass@1 (%), 3,374 problems: 1,687 English and 1,687 Chinese; December 2024 release. Source: github.com. Saturation forecast: Estimated already saturated. 27 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | O1 Preview | 65.62 | #477 |
| 2 | O1 Mini | 64.73 | #760 |
| 3 | Claude 3.5 Sonnet | 62.57 | #755 |
| 4 | GPT-4o (2024-08-06) | 61.77 | #750 |
| 5 | Doubao-Coder-Preview | 58.92 | |
| 6 | DeepSeek Coder V2 Instruct | 58.09 | #992 |
| 7 | Qwen 2.5 72B Instruct | 56.88 | #1124 |
| 8 | Qwen 2.5 Coder 32B Instruct | 56.88 | #1590 |
| 9 | GLM-4 Plus | 56.4 | #1045 |
| 10 | Qwen 2.5 Coder 14B Instruct | 55.28 | #1842 |
| 11 | Qwen Max | 55.16 | #1094 |
| 12 | Llama 3.1 70B Instruct | 51.45 | #1579 |
| 13 | DeepSeek Coder V2 Lite Instruct | 48.73 | #3823 |
| 14 | DeepSeek-Coder-33B-Instruct | 48.61 | #2592 |
| 15 | Qwen 2.5 Coder 7B Instruct | 47.95 | #3976 |
No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=fullstackbench-2024-12-bilingual · How It Works · Data refreshed daily, snapshot 2026-11-22.