EsoBench — leaderboard
Can models learn an unknown programming language from scratch? 30 tasks across 6 difficulty levels with only 1 example and 50 experimental attempts. No model has fully solved it - top score is 29.4/100.
Metric: Score. Source: caseys-evals.com. Status: saturation imminent. 38 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | Qwen 3 235B A22B (Thinking) | 29.4 |
| 2 | GPT-5 | 28.13 |
| 3 | GPT-5.2 (High) | 26.87 |
| 4 | O4 Mini (2025-04-16) (High) | 26.8 |
| 5 | GPT-5 Mini | 23.73 |
| 6 | O4 Mini (2025-04-16) | 23.07 |
| 7 | Grok 4 | 21.4 |
| 8 | Claude Opus 4.5 (20251101) | 20.93 |
| 9 | GPT-5.1 (High) | 19.33 |
| 10 | Claude Opus 4.1 (20250805) | 19.27 |
| 11 | Gemini 3 Pro (Preview) | 17.73 |
| 12 | Claude Sonnet 4.6 | 17.4 |
| 13 | Grok 3 | 16.8 |
| 14 | Claude Opus 4 (20250514) | 16.8 |
| 15 | O3 Mini (2025-01-31) | 16.67 |
Interactive version: theaggregate.ai/benchmark?slug=esobench · How the rankings work · Data refreshed daily, snapshot 2026-07-22.