AetherCode - 2025 Problems: leaderboard

The same runs restricted to the 56 problems from January to May 2025 contests, whose test cases the authors keep private. The score is Pass@1 over four samples per problem, run by the benchmark's authors at ByteDance Seed.

Metric: Pass@1 (%). Source: huggingface.co. Saturation forecast: Around February 2027. 9 models tracked. Updated 17 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1O4 MiniHigh32.6#367
2Seed-1.6-thinking-071514.7#492
3DeepSeek R1 052814.3#431
4Qwen 3 235B A22BThinking7.1#631
5Qwen 3 32BThinking6.7#965
6DeepSeek V3 (0324)5.4#722
7GPT-4.14.5#505
8Qwen 3 8BThinking2.7#1868
9Qwen 3 Coder 480B A35B Instruct1.8#653

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=aethercode-2025-problems · How It Works · Data refreshed daily, snapshot 2026-11-17.