ARMOR 2025: leaderboard

ARMOR 2025 (ICMCIS, 2026) is a military-aligned safety benchmark of 519 three-option multiple-choice questions grounded in the US Law of War, Rules of Engagement and Joint Ethics Regulation, organised into twelve doctrinal categories along the OODA decision loop. The score is accuracy in percent, macro-averaged over the categories, zero-shot at API default settings; higher is better.

Metric: Accuracy (%) macro-averaged over the 12 doctrinal categories of ARMOR 2025 (519 three-option multiple-choice questions derived from the US Law of War, Rules of Engagement and Joint Ethics Regulation and verified by humans), zero-shot at API default parameters; an output that maps to no option or declines counts as a refusal; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 20 models tracked. Updated 9 Nov 2026.

Top models

#ModelScoreOverall rank
1GPT-4o95.4#751
2ARMOR Gemini 2.0 (checkpoint unspecified)95.4
3GPT-4.1 Mini95#789
4DeepSeek V394.8#729
5O4 Mini94.8#368
6GPT-5 Mini94.3#364
7ARMOR LLaMA 4 17B (checkpoint unspecified)94.3
8ARMOR Gemini 2.5 (checkpoint unspecified)94.1
9O3 Mini93.9#546
10Claude 3 Haiku93.3#2663
11Claude Haiku 4.593.1#583
12DeepSeek R192.6#482
13Claude 3.5 Haiku92.1#1580
14ARMOR Mistral Small 24B (checkpoint unspecified)91.8
15Qwen 2.5 7B Turbo Instruct90.9#2617

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=armor-2025 · How It Works · Data refreshed daily, snapshot 2026-11-21.