ARMOR 2025: leaderboard
ARMOR 2025 (ICMCIS, 2026) is a military-aligned safety benchmark of 519 three-option multiple-choice questions grounded in the US Law of War, Rules of Engagement and Joint Ethics Regulation, organised into twelve doctrinal categories along the OODA decision loop. The score is accuracy in percent, macro-averaged over the categories, zero-shot at API default settings; higher is better.
Metric: Accuracy (%) macro-averaged over the 12 doctrinal categories of ARMOR 2025 (519 three-option multiple-choice questions derived from the US Law of War, Rules of Engagement and Joint Ethics Regulation and verified by humans), zero-shot at API default parameters; an output that maps to no option or declines counts as a refusal; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 20 models tracked. Updated 9 Nov 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-4o | 95.4 | #751 |
| 2 | ARMOR Gemini 2.0 (checkpoint unspecified) | 95.4 | |
| 3 | GPT-4.1 Mini | 95 | #789 |
| 4 | DeepSeek V3 | 94.8 | #729 |
| 5 | O4 Mini | 94.8 | #368 |
| 6 | GPT-5 Mini | 94.3 | #364 |
| 7 | ARMOR LLaMA 4 17B (checkpoint unspecified) | 94.3 | |
| 8 | ARMOR Gemini 2.5 (checkpoint unspecified) | 94.1 | |
| 9 | O3 Mini | 93.9 | #546 |
| 10 | Claude 3 Haiku | 93.3 | #2663 |
| 11 | Claude Haiku 4.5 | 93.1 | #583 |
| 12 | DeepSeek R1 | 92.6 | #482 |
| 13 | Claude 3.5 Haiku | 92.1 | #1580 |
| 14 | ARMOR Mistral Small 24B (checkpoint unspecified) | 91.8 | |
| 15 | Qwen 2.5 7B Turbo Instruct | 90.9 | #2617 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=armor-2025 · How It Works · Data refreshed daily, snapshot 2026-11-21.