AthenaBench - VSP: leaderboard
Predict the CVSS v3 vector of 2,000 vulnerabilities; the score is 1 minus the mean absolute error of the resulting base score over 7.7, in percent, averaged over predictions that parse as CVSS vectors (unparseable answers are dropped, not scored 0). It goes negative when the error exceeds 7.7.
Metric: Accuracy (%) = 1 - MAD/7.7 of predicted CVSS base scores over the parseable predictions (2,000 items; negative when MAD exceeds 7.7). Source: raw.githubusercontent.com. Saturation forecast: Around December 2026. 10 models tracked. Updated 17 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Gemini 2.5 Pro | 85.4 | #274 | |
| 2 | GPT-5 | 85.4 | #177 | |
| 3 | Qwen 3 8B | Thinking | 82.6 | #1868 |
| 4 | Qwen 3 14B | Reasoning | 80.3 | #1264 |
| 5 | Qwen 3 4B 2507 Instruct | 79.6 | #2215 | |
| 6 | Gemini 2.5 Flash | 78.5 | #467 | |
| 7 | Llama 3.1 8B Instruct | 74 | #4576 | |
| 8 | Llama-Primus-Merged | 71.9 | #2461 | |
| 9 | Llama 3.3 70B Instruct | 70.1 | #1387 | |
| 10 | Llama 3 70B Instruct | 63.8 | #1689 |
No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.
Interactive version: theaggregate.ai/benchmark?slug=athenabench-vsp · How It Works · Data refreshed daily, snapshot 2026-11-17.