IFMTBench - Single-Constraint Translation Quality: leaderboard

xCOMET-XXL translation quality (%) on the 4,506 single-constraint items in IFMTBench (2026), a benchmark of translation requests into seven languages that carry constraints such as schema preservation, glossaries, code spans and register. It measures fidelity to the reference regardless of the constraints; higher is better.

Metric: xCOMET-XXL translation quality (%) on the 4,506 single-constraint items, translation requests into seven languages with instructions paraphrased in all seven, officially recommended decoding with non-thinking mode for open models; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 15 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Hy-MT2-30B-A3B85.55
2Hy-MT2-7B84.29
3Gemma 4 31B83.56#382
4Gemini 3.1 Pro (Preview)83.31#99
5Gemma 4 26B A4B83.3#590
6Qwen 3.5 27BNon-reasoning82.64#375
7Qwen 3.6 35B A3BNon-reasoning82.56#481
8Gemma 4 E4B82.26#2184
9Qwen 3.5 35B A3BNon-reasoning81.74#492
10Qwen 3.5 9BNon-reasoning80.8#828
11Hy-MT2-1.8B80.74
12Gemma 4 E2B80.22#4317
13Qwen 3.5 4BNon-reasoning80.16#1241
14Qwen 3.5 2BNon-reasoning75.77#5307
15Qwen 3.5 0.8BNon-reasoning70.21#8232

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=ifmtbench-single-constraint-translation-quality · How It Works · Data refreshed daily, snapshot 2026-11-21.