Orak - Minecraft: leaderboard
Orak Minecraft: an LLM agent writes Mineflayer JavaScript to gather resources and craft target items; score is whether the target item ends up in the inventory, averaged over tasks and trials as 0–100, with the game's default agentic strategy. Higher is better. From KRAFTON AI's Orak benchmark (NeurIPS 2025).
Metric: Game Score (0-100). Source: krafton-ai.github.io. Saturation forecast: Estimated already saturated. 12 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 2.5 Pro (Preview 03-25) | 75 | #126 |
| 2 | Claude 3.7 Sonnet | 75 | #259 |
| 3 | O3 Mini (2025-01-31) | 75 | #249 |
| 4 | GPT-4o (2024-08-06) | 71 | #343 |
| 5 | GPT-4o Mini | 46 | #598 |
| 6 | DeepSeek R1 | 41.7 | #262 |
| 7 | Qwen 2.5 7B Instruct | 0 | #863 |
| 8 | Llama 3.2 3B Instruct | 0 | #1327 |
| 9 | Llama 3.2 1B Instruct | 0 | #1518 |
| 10 | Qwen 2.5 3B Instruct | 0 | #1159 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=orak-minecraft · How It Works · Data refreshed daily, snapshot 2026-11-05.