M-ALERT: leaderboard
The model answers 15k red-teaming prompts from ALERT in each of English, German, Spanish, French and Italian; LlamaGuard-3 labels each answer safe or unsafe and the score is the paper's printed mean of the five languages' safe shares, given for 13 base models and their Instruct versions. Unlike ALERT, it tests whether safety holds across translations of the same prompts.
Metric: Safe responses, mean of five languages as printed in Table 6 (%, higher is better). Source: arxiv.org. 26 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemma 2 27B (IT) | 99.05 | #1782 |
| 2 | Gemma 2 9B (IT) | 99.04 | #2262 |
| 3 | Gemma 2 2B (IT) | 98.74 | #6876 |
| 4 | Llama 3.1 8B Instruct | 98.71 | #4246 |
| 5 | Qwen 2.5 14B Instruct | 98.68 | #1732 |
| 6 | Qwen 2.5 32B Instruct | 98.35 | #1215 |
| 7 | Qwen 2.5 72B Instruct | 98.33 | #1092 |
| 8 | Qwen 2.5 3B Instruct | 97.85 | #5150 |
| 9 | Qwen 2.5 7B Instruct | 97.6 | #2564 |
| 10 | Llama 3.2 3B Instruct | 97.43 | #7147 |
| 11 | Llama 3 8B Instruct | 96.66 | #4612 |
| 12 | Qwen 2.5 1.5B Instruct | 95.81 | #7470 |
| 13 | Qwen 2.5 32B | 88.02 | #1332 |
| 14 | Qwen 2.5 0.5B Instruct | 87.53 | #8987 |
| 15 | Qwen 2.5 14B | 87.06 | #1785 |
Interactive version: theaggregate.ai/benchmark?slug=m-alert · How It Works · Data refreshed daily, snapshot 2026-11-13.