KoALa-Bench - SIF Vicuna: leaderboard
Metric: Speech instruction-following score (GPT-4o judge returning 0-1 against the reference answer, x100) on the Korean spoken Vicuna instructions, Korean speech synthesized or recorded per item, clean audio (first value of each clean / noisy pair), mean over four prompts with greedy decoding; higher is better. Source: arxiv.org. Saturation forecast: Around January 2027. 6 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | Gemma 3n E4B (IT) | 79.38 |
| 2 | Qwen3 Omni 30B A3B Instruct | 76.5 |
| 3 | Gemini 2.5 Flash Lite | 76.43 |
| 4 | Voxtral-Mini-3B-2507 | 62.82 |
Interactive version: theaggregate.ai/benchmark?slug=koala-bench-sif-vicuna · How It Works · Data refreshed daily, snapshot 2026-10-07.