IFMTBench - Multi-Constraint: leaderboard

Instruction-following score (%) on the 2,838 items that combine several constraints in IFMTBench (2026), a benchmark of translation requests into seven languages that carry constraints such as schema preservation, glossaries, code spans and register. Hard constraints are checked by deterministic verifiers and gate the item, soft ones are rated by an LLM judge; higher is better.

Metric: IF_Score (%): hard constraints checked by deterministic verifiers gate the item, soft constraints are scored by a gpt-oss-120b rubric judge, mean over items, on the 2,838 multi-constraint items, translation requests into seven languages with instructions paraphrased in all seven, officially recommended decoding with non-thinking mode for open models; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 15 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Gemini 3.1 Pro (Preview)84.53#102
2Qwen 3.5 27BNon-reasoning78.81#380
3Hy-MT2-30B-A3B75.8
4Gemma 4 26B A4B73.89#602
5Hy-MT2-7B72.67
6Gemma 4 31B72.64#390
7Qwen 3.6 35B A3BNon-reasoning71.43#485
8Qwen 3.5 35B A3BNon-reasoning70.32#494
9Gemma 4 E4B69.25#2199
10Qwen 3.5 9BNon-reasoning64.28#838
11Hy-MT2-1.8B57.61
12Qwen 3.5 4BNon-reasoning57.23#1270
13Gemma 4 E2B50.72#4202
14Qwen 3.5 2BNon-reasoning24.6#5380
15Qwen 3.5 0.8BNon-reasoning7.46#8198

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=ifmtbench-multi-constraint · How It Works · Data refreshed daily, snapshot 2026-11-22.