AetherCode: leaderboard
Each model writes solutions to 456 problems from 2024 and January-May 2025 olympiad and ICPC contests, sampled four times per problem with up to 32,768 output tokens; the benchmark's authors at ByteDance Seed ran every model, including their own Seed-1.6-thinking. The score is Pass@1, the average share of samples that pass test suites validated against more than 30,000 human submissions; the problems come from premier contests rather than Codeforces or LeetCode as in LiveCodeBench.
Metric: Pass@1 (%). Source: huggingface.co. Saturation forecast: Around February 2027. 9 models tracked. Updated 17 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | O4 Mini | High | 35.5 | #367 |
| 2 | Seed-1.6-thinking-0715 | 26.6 | #492 | |
| 3 | DeepSeek R1 0528 | 22.3 | #431 | |
| 4 | Qwen 3 235B A22B | Thinking | 17.6 | #631 |
| 5 | Qwen 3 32B | Thinking | 16.3 | #965 |
| 6 | GPT-4.1 | 10.5 | #505 | |
| 7 | Qwen 3 8B | Thinking | 10 | #1868 |
| 8 | DeepSeek V3 (0324) | 8.5 | #722 | |
| 9 | Qwen 3 Coder 480B A35B Instruct | 7.7 | #653 |
No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.
Interactive version: theaggregate.ai/benchmark?slug=aethercode · How It Works · Data refreshed daily, snapshot 2026-11-17.