ArabCulture-Dialogue - MCQ with Country Context (MSA): leaderboard
ArabCulture-Dialogue (MBZUAI, 2026) tests cultural reasoning in human-curated multi-turn dialogues from 13 Arab countries, each available in Modern Standard Arabic and in the country dialect. The model picks the culturally appropriate continuation out of three by option likelihood, here on the Modern Standard Arabic dialogues with the region and country named in the prompt; accuracy in percent with chance at 33.3, higher is better.
Metric: Accuracy (%) selecting the one culturally appropriate continuation of three for each dialogue of ArabCulture-Dialogue (13 Arab countries, parallel Modern Standard Arabic and country-dialect versions of 3,471 culturally grounded multi-turn dialogues built from ArabCulture), MSA dialogues, with the region and country named in the prompt, scored by likelihood of the options, single run; chance is 33.3; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 12 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5 | 95.3 | #178 |
| 2 | Gemini 2.5 Pro | 95 | #276 |
| 3 | Hala-9B | 82 | #1162 |
| 4 | SILMA-9B-Instruct-v1.0 | 78.4 | #3847 |
| 5 | Jais-2-8B-Chat | 76.6 | #2496 |
| 6 | Gemma 2 9B (IT) | 71 | #2369 |
| 7 | Fanar-1-9B | 61.8 | #1913 |
| 8 | jais-adapted-7B-chat | 53.1 | #5831 |
| 9 | ALLaM-7B-Instruct-preview | 50.6 | #2001 |
| 10 | Llama 3.1 8B Instruct | 47.3 | #4600 |
| 11 | Qwen 3 8B | 38 | #1873 |
| 12 | c4ai-command-r7B-arabic-02-2025 | 37.5 | #2135 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=arabculture-dialogue-mcq-with-country-context-msa · How It Works · Data refreshed daily, snapshot 2026-11-21.