AthenaBench - RMS: leaderboard

Propose ATT&CK mitigations for a described attack, 500 items; mean F1 of the mitigation IDs against the reference set.

Metric: F1 (%) of proposed ATT&CK mitigations against the reference set (500 items). Source: raw.githubusercontent.com. Saturation forecast: Around March 2027. 9 models tracked. Updated 17 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1Gemini 2.5 Pro28.4#274
2Gemini 2.5 Flash13.4#467
3Llama 3.3 70B Instruct11.1#1387
4Llama 3 70B Instruct11.1#1689
5Qwen 3 14BReasoning7#1264
6Llama-Primus-Merged6.6#2461
7Qwen 3 8BThinking5.5#1868
8Qwen 3 4B 2507 Instruct4.8#2215
9Llama 3.1 8B Instruct3.6#4576

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=athenabench-rms · How It Works · Data refreshed daily, snapshot 2026-11-17.