LiT (Lost in Translation): leaderboard
LiT, Lost in Translation (2026), measures multilingual proficiency by round-trip translation: English passages pass through a sequence of four languages and back, and an LLM judge scores the result against the source with MQM error weights. Share of passages, averaged over the eight categories of paper abstracts, pragmatics and informal text, whose round trip scores at least 80, in percent; higher is better.
Metric: MQM>=80 rate (%): share of round-trip translations whose back-translation to English scores at least 80 on an MQM error scale (minor -1, major -5, critical -25) from a Grok 4.1 Fast judge comparing it with the English source; each passage is translated serially through a four-language sequence and back, averaged over eight sequences (high, medium and low-resource languages); mean of the eight category rates over all 200 LiT passages (abstracts, pragmatics and informal text); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 17 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Gemini 3 Flash | Non-reasoning | 87.2 | #175 |
| 2 | Qwen 3.5 397B A17B | 73.8 | #264 | |
| 3 | Gemma 4 31B (IT) | 73 | #350 | |
| 4 | GLM-5 | 70.3 | #269 | |
| 5 | Kimi K2 (Thinking) | 64.7 | ||
| 6 | GLM-4.7 | 61.6 | #380 | |
| 7 | Qwen 3 235B A22B 2507 (Thinking) | 55.6 | #466 | |
| 8 | DeepSeek V3.2 Exp | Non-reasoning | 53.4 | #439 |
| 9 | Qwen 3.5 35B A3B | 52 | #492 | |
| 10 | Kimi K2 | 49.7 | #443 | |
| 11 | Gemma 3 27B (IT) | 47.8 | #1324 | |
| 12 | Qwen 3 235B A22B 2507 Instruct | 44.1 | #616 | |
| 13 | GPT-OSS-120B | High | 41.9 | #704 |
| 14 | MiniMax-M2.5 | 39.9 | #600 | |
| 15 | MiMo-V2-Flash | Non-reasoning | 30.2 | #684 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=lit-lost-in-translation · How It Works · Data refreshed daily, snapshot 2026-11-21.