IFMTBench - Multi-Constraint: leaderboard
Instruction-following score (%) on the 2,838 items that combine several constraints in IFMTBench (2026), a benchmark of translation requests into seven languages that carry constraints such as schema preservation, glossaries, code spans and register. Hard constraints are checked by deterministic verifiers and gate the item, soft ones are rated by an LLM judge; higher is better.
Metric: IF_Score (%): hard constraints checked by deterministic verifiers gate the item, soft constraints are scored by a gpt-oss-120b rubric judge, mean over items, on the 2,838 multi-constraint items, translation requests into seven languages with instructions paraphrased in all seven, officially recommended decoding with non-thinking mode for open models; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 15 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) | 84.53 | #102 | |
| 2 | Qwen 3.5 27B | Non-reasoning | 78.81 | #380 |
| 3 | Hy-MT2-30B-A3B | 75.8 | ||
| 4 | Gemma 4 26B A4B | 73.89 | #602 | |
| 5 | Hy-MT2-7B | 72.67 | ||
| 6 | Gemma 4 31B | 72.64 | #390 | |
| 7 | Qwen 3.6 35B A3B | Non-reasoning | 71.43 | #485 |
| 8 | Qwen 3.5 35B A3B | Non-reasoning | 70.32 | #494 |
| 9 | Gemma 4 E4B | 69.25 | #2199 | |
| 10 | Qwen 3.5 9B | Non-reasoning | 64.28 | #838 |
| 11 | Hy-MT2-1.8B | 57.61 | ||
| 12 | Qwen 3.5 4B | Non-reasoning | 57.23 | #1270 |
| 13 | Gemma 4 E2B | 50.72 | #4202 | |
| 14 | Qwen 3.5 2B | Non-reasoning | 24.6 | #5380 |
| 15 | Qwen 3.5 0.8B | Non-reasoning | 7.46 | #8198 |
No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=ifmtbench-multi-constraint · How It Works · Data refreshed daily, snapshot 2026-11-22.