Orak - Pokemon Red: leaderboard
Orak Pokémon Red: an LLM agent explores and battles in Pokémon Red; score is the percentage of 12 storyline flags reached, from exiting Red's house to defeating Pewter Gym leader Brock (0–100), averaged over trials with the game's default agentic strategy. Higher is better. From KRAFTON AI's Orak benchmark (NeurIPS 2025).
Metric: Game Score (0-100). Source: krafton-ai.github.io. Saturation forecast: Estimated already saturated. 12 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 2.5 Pro (Preview 03-25) | 83.3 | #126 |
| 2 | DeepSeek R1 | 75 | #262 |
| 3 | Claude 3.7 Sonnet | 63.9 | #259 |
| 4 | GPT-4o (2024-08-06) | 38.9 | #343 |
| 5 | GPT-4o Mini | 0 | #598 |
| 6 | Qwen 2.5 7B Instruct | 0 | #863 |
| 7 | Llama 3.2 3B Instruct | 0 | #1327 |
| 8 | Llama 3.2 1B Instruct | 0 | #1518 |
| 9 | O3 Mini (2025-01-31) | 0 | #249 |
| 10 | Qwen 2.5 3B Instruct | 0 | #1159 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=orak-pokemon-red · How It Works · Data refreshed daily, snapshot 2026-11-05.