IFMTBench - Multi-Constraint Translation Quality: leaderboard

xCOMET-XXL translation quality (%) on the 2,838 multi-constraint items in IFMTBench (2026), a benchmark of translation requests into seven languages that carry constraints such as schema preservation, glossaries, code spans and register. It measures fidelity to the reference regardless of the constraints; higher is better.

Metric: xCOMET-XXL translation quality (%) on the 2,838 multi-constraint items, translation requests into seven languages with instructions paraphrased in all seven, officially recommended decoding with non-thinking mode for open models; higher is better. Source: arxiv.org. Saturation forecast: Around September 2028. 15 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Hy-MT2-30B-A3B74.79
2Hy-MT2-7B72.76
3Gemini 3.1 Pro (Preview)69.65#99
4Gemma 4 26B A4B69.61#590
5Gemma 4 31B69.5#382
6Qwen 3.5 27BNon-reasoning69.29#375
7Qwen 3.6 35B A3BNon-reasoning68.91#481
8Qwen 3.5 35B A3BNon-reasoning68.48#492
9Hy-MT2-1.8B67.8
10Gemma 4 E4B67.77#2184
11Qwen 3.5 9BNon-reasoning67.62#828
12Gemma 4 E2B66.92#4317
13Qwen 3.5 4BNon-reasoning66.44#1241
14Qwen 3.5 2BNon-reasoning61.83#5307
15Qwen 3.5 0.8BNon-reasoning52.74#8232

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=ifmtbench-multi-constraint-translation-quality · How It Works · Data refreshed daily, snapshot 2026-11-21.