LiveChatBench - ChrF++: leaderboard
The LiveChatBench Korean-to-English live-stream chat set (Park et al., 2026; 1,000 messages full of memes and slang, human reference translations) scored by character n-gram F-score. ChrF++ on a 0-100 scale, higher is better; the on-device fine-tunes on the board were trained on synthetic pairs from the same platform.
Metric: ChrF++ (0-100; Korean-to-English translation of 1,000 live-stream chat messages against human references). Source: arxiv.org. Saturation forecast: Around March 2027. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.1 | 45.26 | #225 |
| 2 | Gemma 3 1B (IT) | 17.56 | #8752 |
| 3 | Qwen 3 0.6B | 8.65 | #8392 |
| 4 | Gemma 3 270M (IT) | 5.39 | #9460 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=livechatbench-chrf-plus-plus · How It Works · Data refreshed daily, snapshot 2026-11-12.