Diplomacy: Overall Performance: leaderboard
20 games against Devstral Small 1.1; source metric combines game duration and supply centers. Current live aggregation implementation/horizon not fully corroborated; do not infer percentage or a finite maximum.
Metric: Mean native game score (duration and supply centers), against Devstral Small 1.1. Source: goodstartlabs.com. Saturation forecast: Around January 2027. 23 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3 Pro | 60.3 | #80 |
| 2 | O3 | 58.42 | #124 |
| 3 | Grok 4 Fast (Reasoning) | 57.75 | #254 (Grok 4 Fast) |
| 4 | Claude Opus 4.5 | 57.45 | #83 |
| 5 | Gemini 2.5 Flash | 57.3 | #245 |
| 6 | Gemini 2.5 Pro | 55.65 | #152 |
| 7 | Claude Sonnet 4.5 | 55.05 | #142 |
| 8 | GPT-5 (Minimal) | 54 | #98 (GPT-5) |
| 9 | GLM-4.6 | 53.4 | #252 |
| 10 | Kimi K2 | 53.25 | #243 |
| 11 | GPT-5 Mini | 53.12 | #186 |
| 12 | Grok 4 Fast | 51.25 | #254 |
| 13 | Claude Haiku 4.5 | 49.85 | #280 |
| 14 | Hermes 4 405B | 48.8 | #334 |
| 15 | GPT-5 (Medium) | 47.4 | #98 (GPT-5) |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=diplomacy-overall-performance · How It Works · Data refreshed daily, snapshot 2026-10-20.