JetBrains Mellum2.1 Card - IFEval: leaderboard
IFEval instruction following. JetBrains evaluated Mellum2.1 Thinking, Mellum2 Thinking, Gemma 4 E4B and Qwen3.5 9B with the same pipeline in thinking mode (greedy decoding) and reported the results on the Mellum2.1 model card (October 2026). Vendor-reported. Metric: Accuracy (%). Higher is better. Source: https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking/raw/92ddae9fc7665e9f801d141d2e5a6b2caf2460c4/README.md
Metric: Accuracy (%). Source: huggingface.co. Saturation forecast: Around January 2027. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3.5 9B (Thinking) | 92.4 | #377 (Qwen 3.5 9B) |
| 2 | Gemma 4 E4B (IT) (Thinking) | 90.8 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=jetbrains-mellum2-1-card-ifeval · How It Works · Data refreshed daily, snapshot 2026-10-19.