KoALa-Bench - SIF OpenHermes: leaderboard
KoALa-Bench (2026) evaluates large audio language models on Korean speech understanding and on faithfulness to the speech context, under clean and noisy audio. This board is the GPT-4o-judged instruction-following score, on a 0 to 100 scale, for spoken Korean OpenHermes instructions, clean audio; higher is better.
Metric: Speech instruction-following score (GPT-4o judge returning 0-1 against the reference answer, x100) on the Korean spoken OpenHermes instructions, Korean speech synthesized or recorded per item, clean audio (first value of each clean / noisy pair), mean over four prompts with greedy decoding; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 6 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-Audio Mini | 89.42 | #918 |
| 2 | Qwen3 Omni 30B A3B Instruct | 84.09 | #930 |
| 3 | Gemma 3n E4B (IT) | 82.93 | #2401 |
| 4 | Gemini 2.5 Flash Lite | 82.69 | #964 |
| 5 | Raon-Speech-9B | 78.65 | |
| 6 | Voxtral-Mini-3B-2507 | 65.82 | #3682 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=koala-bench-sif-openhermes · How It Works · Data refreshed daily, snapshot 2026-11-21.