SEABED - Dialect Identification (Open-Ended): leaderboard
SEABED (Rajgarhia et al., 2026) is an audio-first reasoning benchmark on Southeast Asian speech built from openly available corpora. In this task the model hears four concatenated utterances in different Indonesian or Thai lects and answers questions about which lect was spoken where; answers are open-ended and a Grok 4.5 judge marks each answer correct or incorrect, and the score is accuracy in percent, higher is better.
Metric: Accuracy (%; 504 dialect and language identification over four concatenated Indonesian or Thai lects items; open-ended answers judged by Grok 4.5, binary correct or incorrect (graded 0 to 1 for long-form); audio at 16 kHz). Source: arxiv.org. Saturation forecast: Around January 2028. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.5 Flash | 34.1 | #98 |
| 2 | Gemini 3.6 Flash | 32.5 | #72 |
| 3 | Gemini 2.5 Pro | 29.1 | #262 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=seabed-dialect-identification-open-ended · How It Works · Data refreshed daily, snapshot 2026-11-12.