FullStackBench (2024-12 bilingual): leaderboard
Metric: Pass@1 (%), 3,374 problems: 1,687 English and 1,687 Chinese; December 2024 release. Source: github.com. Saturation forecast: Estimated already saturated. 27 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | O1 Preview | 65.62 |
| 2 | O1 Mini | 64.73 |
| 3 | Claude 3.5 Sonnet | 62.57 |
| 4 | GPT-4o (2024-08-06) | 61.77 |
| 5 | Qwen 2.5 72B Instruct | 56.88 |
| 6 | Qwen 2.5 Coder 32B Instruct | 56.88 |
| 7 | GLM-4 Plus | 56.4 |
| 8 | Qwen 2.5 Coder 14B Instruct | 55.28 |
| 9 | Qwen Max | 55.16 |
| 10 | Llama 3.1 70B Instruct | 51.45 |
| 11 | Qwen 2.5 Coder 7B Instruct | 47.95 |
| 12 | Yi-Coder-9B-Chat | 47.13 |
| 13 | OpenCoder-8B-Instruct | 43.63 |
| 14 | deepseek-coder-6.7B-instruct | 41.88 |
| 15 | CodeQwen1.5-7B Chat | 40.52 |
Interactive version: theaggregate.ai/benchmark?slug=fullstackbench-2024-12-bilingual · How It Works · Data refreshed daily, snapshot 2026-10-08.