ALPHA CWE Penalty (SecurityEval): leaderboard
Metric: ALPHA penalty (sum over 121 vulnerable Python functions of hierarchy distance x direction multiplier between the predicted and true CWE; 0 is perfect; mean of three runs at temperature 0.8). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 7 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | devstral:24b [Q4_K_M] | 578.5 |
| 2 | qwen2.5-coder:32b [Q4_K_M] | 617.8 |
| 3 | phi4:14b [Q4_K_M] | 797.4 |
| 4 | llama3.1:8b [Q4_K_M] | 856.5 |
| 5 | qwen2.5-coder:7b [Q4_K_M] | 921.5 |
| 6 | mistral:7b [Q4_K_M] | 1156.9 |
| 7 | deepseek-coder:6.7b [Q4_0] | 1228.5 |
Interactive version: theaggregate.ai/benchmark?slug=alpha-cwe-penalty-securityeval · How It Works · Data refreshed daily, snapshot 2026-09-25.