ArabicDialectSafety - Unsafe Response Rate: leaderboard
ArabicDialectSafety (Zaghouani et al., 2026) is a human-curated set of about 24,000 Arabic prompts in Modern Standard Arabic and five dialects, annotated for safety and seven harm categories. This board samples 500 harmful dialectal Arabic test prompts and has a GPT-5.2 judge label each model response safe or unsafe. Percentage of unsafe responses; lower is better.
Metric: Unsafe response rate (%; share of 500 stratified harmful dialectal Arabic test prompts whose response a GPT-5.2 judge labels unsafe under a binary rubric; lower is better). Source: arxiv.org. Saturation forecast: Estimated already saturated. 7 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3.6 Plus | 0.2 | #217 |
| 2 | Claude Sonnet 4.6 | 0.6 | #158 |
| 3 | Claude 3 Haiku | 1.6 | #2629 |
| 4 | Gemini 2.5 Flash | 2 | #452 |
| 5 | GPT-4o Mini | 4.6 | #1627 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=arabicdialectsafety-unsafe-response-rate · How It Works · Data refreshed daily, snapshot 2026-11-12.