AetherCode - 2025 Problems: leaderboard
The same runs restricted to the 56 problems from January to May 2025 contests, whose test cases the authors keep private. The score is Pass@1 over four samples per problem, run by the benchmark's authors at ByteDance Seed.
Metric: Pass@1 (%). Source: huggingface.co. Saturation forecast: Around February 2027. 9 models tracked. Updated 17 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | O4 Mini | High | 32.6 | #367 |
| 2 | Seed-1.6-thinking-0715 | 14.7 | #492 | |
| 3 | DeepSeek R1 0528 | 14.3 | #431 | |
| 4 | Qwen 3 235B A22B | Thinking | 7.1 | #631 |
| 5 | Qwen 3 32B | Thinking | 6.7 | #965 |
| 6 | DeepSeek V3 (0324) | 5.4 | #722 | |
| 7 | GPT-4.1 | 4.5 | #505 | |
| 8 | Qwen 3 8B | Thinking | 2.7 | #1868 |
| 9 | Qwen 3 Coder 480B A35B Instruct | 1.8 | #653 |
No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.
Interactive version: theaggregate.ai/benchmark?slug=aethercode-2025-problems · How It Works · Data refreshed daily, snapshot 2026-11-17.