LiveChatBench: leaderboard
Korean-to-English translation of live-stream chat (LiveChatBench; Park et al., 2026): 1,000 short chat messages from a Korean streaming platform, dense with memes, slang and game jargon, with human reference translations. BLEU on a 0-100 scale, higher is better; the on-device fine-tunes on the board were trained on 1.5 million synthetic pairs from the same platform.
Metric: BLEU (0-100; Korean-to-English translation of 1,000 live-stream chat messages against human references; printed as a fraction). Source: arxiv.org. Saturation forecast: Around June 2027. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.1 | 26.79 | #225 |
| 2 | Gemma 3 1B (IT) | 4.3 | #8752 |
| 3 | Gemma 3 270M (IT) | 0.97 | #9460 |
| 4 | Qwen 3 0.6B | 0.17 | #8392 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=livechatbench · How It Works · Data refreshed daily, snapshot 2026-11-12.