AthenaBench - ATE: leaderboard

Name the MITRE ATT&CK technique a threat-report passage describes, 500 items; accuracy on the top-level technique ID (sub-technique suffix ignored).

Metric: Accuracy (%) extracting ATT&CK techniques from threat reports (500 items). Source: raw.githubusercontent.com. Saturation forecast: Estimated already saturated. 9 models tracked. Updated 17 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1Gemini 2.5 Pro76.2#274
2Gemini 2.5 Flash51.6#467
3Llama-Primus-Merged33.8#2461
4Llama 3 70B Instruct31.6#1689
5Llama 3.3 70B Instruct30.4#1387
6Qwen 3 14BReasoning19.4#1264
7Llama 3.1 8B Instruct16.4#4576
8Qwen 3 8BThinking11.8#1868
9Qwen 3 4B 2507 Instruct5.6#2215

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=athenabench-ate · How It Works · Data refreshed daily, snapshot 2026-11-17.