KoALa-Bench - SIF OpenHermes: leaderboard
Metric: Speech instruction-following score (GPT-4o judge returning 0-1 against the reference answer, x100) on the Korean spoken OpenHermes instructions, Korean speech synthesized or recorded per item, clean audio (first value of each clean / noisy pair), mean over four prompts with greedy decoding; higher is better. Source: arxiv.org. Saturation forecast: Around January 2027. 6 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | Qwen3 Omni 30B A3B Instruct | 84.09 |
| 2 | Gemma 3n E4B (IT) | 82.93 |
| 3 | Gemini 2.5 Flash Lite | 82.69 |
| 4 | Voxtral-Mini-3B-2507 | 65.82 |
Interactive version: theaggregate.ai/benchmark?slug=koala-bench-sif-openhermes · How It Works · Data refreshed daily, snapshot 2026-10-07.