IFMTBench - Single-Constraint: leaderboard
Instruction-following score (%) on the 4,506 single-constraint translation items in IFMTBench (2026), a benchmark of translation requests into seven languages that carry constraints such as schema preservation, glossaries, code spans and register. Hard constraints are checked by deterministic verifiers and gate the item, soft ones are rated by an LLM judge; higher is better.
Metric: IF_Score (%): hard constraints checked by deterministic verifiers gate the item, soft constraints are scored by a gpt-oss-120b rubric judge, mean over items, on the 4,506 single-constraint items, translation requests into seven languages with instructions paraphrased in all seven, officially recommended decoding with non-thinking mode for open models; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 15 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) | 91.95 | #102 | |
| 2 | Hy-MT2-30B-A3B | 91.94 | ||
| 3 | Hy-MT2-7B | 89.73 | ||
| 4 | Gemma 4 31B | 84 | #390 | |
| 5 | Gemma 4 26B A4B | 83.17 | #602 | |
| 6 | Qwen 3.5 27B | Non-reasoning | 82.48 | #380 |
| 7 | Qwen 3.6 35B A3B | Non-reasoning | 77.95 | #485 |
| 8 | Hy-MT2-1.8B | 76.76 | ||
| 9 | Qwen 3.5 35B A3B | Non-reasoning | 76.46 | #494 |
| 10 | Qwen 3.5 9B | Non-reasoning | 75.59 | #838 |
| 11 | Gemma 4 E4B | 74.64 | #2199 | |
| 12 | Qwen 3.5 4B | Non-reasoning | 72.58 | #1270 |
| 13 | Gemma 4 E2B | 63.71 | #4202 | |
| 14 | Qwen 3.5 2B | Non-reasoning | 54.77 | #5380 |
| 15 | Qwen 3.5 0.8B | Non-reasoning | 40.92 | #8198 |
No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=ifmtbench-single-constraint · How It Works · Data refreshed daily, snapshot 2026-11-22.