CultureTalk-ID - Local Language Dialogue MCQ: leaderboard
CultureTalk-ID (2026) tests cultural commonsense in dialogues grounded in regional Indonesian cultures, in Indonesian and in local languages such as Javanese, Minangkabau and Balinese. This board asks for the culturally appropriate final utterance of the same dialogues in their local language from three options, with no added context; accuracy in percent, higher is better.
Metric: Accuracy (%; the same dialogues in their local languages (Javanese dialects, Acehnese, Minangkabau, Balinese, Banjarese, Buginese, Uab Meto and Wamesa); three-option choice of the culturally appropriate final utterance of a dialogue grounded in a regional Indonesian culture, no province or language context in the prompt, averaged over province-specific and general items (test split); higher is better). Source: arxiv.org. Saturation forecast: Around February 2027. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 2.5 Flash | 87.77 | #453 |
| 2 | GPT-5.1 | 82.31 | #229 |
| 3 | Command A | 75.09 | #1274 |
| 4 | Sailor2-8B-Chat | 70.15 | #2396 |
| 5 | gemma2-9B-cpt-sahabatai-v1-instruct | 69.08 | #1488 |
| 6 | Gemma 2 9B (IT) | 65.17 | #2262 |
| 7 | Qwen 3 8B | 52.89 | #1830 |
| 8 | Llama 3.1 8B Instruct | 51.87 | #4246 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=culturetalk-id-local-language-dialogue-mcq · How It Works · Data refreshed daily, snapshot 2026-11-13.