ALPHA CWE Penalty (SVEN): leaderboard
Metric: ALPHA penalty (sum over 342 vulnerable Python functions of hierarchy distance x direction multiplier between the predicted and true CWE; 0 is perfect; mean of three runs at temperature 0.8). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 7 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | qwen2.5-coder:32b [Q4_K_M] | 574.7 |
| 2 | phi4:14b [Q4_K_M] | 646.4 |
| 3 | devstral:24b [Q4_K_M] | 695.1 |
| 4 | qwen2.5-coder:7b [Q4_K_M] | 753.2 |
| 5 | llama3.1:8b [Q4_K_M] | 813.5 |
| 6 | mistral:7b [Q4_K_M] | 923.3 |
| 7 | deepseek-coder:6.7b [Q4_0] | 2557 |
Interactive version: theaggregate.ai/benchmark?slug=alpha-cwe-penalty-sven · How It Works · Data refreshed daily, snapshot 2026-09-25.