CyberCertBench - Fortinet ICS/SCADA: leaderboard

CyberCertBench (2026) certification knowledge test of 43 Fortinet ICS/SCADA questions (conceptual operational-technology security), asked as 5-shot multiple choice at temperature 0. Exact accuracy (%): a question counts only when all correct options and no wrong one are chosen; higher is better.

Metric: Accuracy (%) on 43 Fortinet ICS/SCADA questions (conceptual operational-technology security): 5-shot multiple-choice question answering at temperature 0, scored by exact accuracy (correct only when the model selects all correct options and no incorrect one), mean accuracy (%) as printed in the appendix; questions scraped from public certification question banks and checked against authoritative documentation; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 45 models tracked. Updated 3 Nov 2026.

Top models

#ModelScoreOverall rank
1GPT-588.4#185
2GPT-OSS-120B88.4#712
3Mistral Small 3.188.4#1734
4Qwen 2.5 72B87.4#1189
5GPT-4o86.5#754
6GPT-4.186#506
7Mistral Large 2 (Nov) Instruct (2411)86#1224
8Gemini 2.5 Pro (Preview 05-06)86#278
9Qwen 2 72B86#1028
10Claude Haiku 4.583.7#587
11GPT-5 Mini83.7#373
12Kimi K283.7#448
13Claude 3.5 Sonnet83.7#755
14Qwen 3 32B81.4#983
15Gemini 2.5 Flash Lite81.4#972

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=cybercertbench-fortinet-ics-scada · How It Works · Data refreshed daily, snapshot 2026-11-22.