Orak - Super Mario: leaderboard
Orak Super Mario: an LLM agent chooses Mario's jump level while he runs right through a level; score is the share of the distance from start to the flag covered before dying (0–100, higher is better), averaged over trials with the game's default agentic strategy. From KRAFTON AI's Orak benchmark (NeurIPS 2025).
Metric: Game Score (0-100). Source: krafton-ai.github.io. Saturation forecast: Estimated already saturated. 12 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 2.5 Pro (Preview 03-25) | 38 | #126 |
| 2 | O3 Mini (2025-01-31) | 34.9 | #249 |
| 3 | GPT-4o (2024-08-06) | 34.1 | #343 |
| 4 | Llama 3.2 3B Instruct | 31.8 | #1327 |
| 5 | Claude 3.7 Sonnet | 31.7 | #259 |
| 6 | GPT-4o Mini | 28.8 | #598 |
| 7 | DeepSeek R1 | 28.7 | #262 |
| 8 | Qwen 2.5 7B Instruct | 27.2 | #863 |
| 9 | Qwen 2.5 3B Instruct | 23.4 | #1159 |
| 10 | Llama 3.2 1B Instruct | 18.7 | #1518 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=orak-super-mario · How It Works · Data refreshed daily, snapshot 2026-11-05.