MathIF: leaderboard
Models solve 420 math problems (GSM8K to AIME) that each carry one to three Python-checkable instructions on length, wording, format or how the answer starts and ends; the score is the share of questions where every instruction is met, one sample per question. Unlike IFEval the instructions ride on math problems the model must also solve.
Metric: Hard accuracy (%): share of 420 constrained math questions whose every constraint is satisfied, 1 sample per question, T=1.0, top-p 0.95, 16k tokens. Source: raw.githubusercontent.com. Saturation forecast: Estimated already saturated. 23 models tracked. Updated 17 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Qwen 3 14B | Reasoning | 50.71 | #1264 |
| 2 | Qwen 3 4B | Reasoning | 44.05 | #2337 |
| 3 | Qwen 3 32B | Thinking | 43.81 | #965 |
| 4 | DeepSeek R1 Distill Qwen 32B | 42.62 | #1806 | |
| 5 | DeepSeek R1 Distill Llama 70B | 41.43 | #1669 | |
| 6 | QwQ-32B | 40.24 | #908 | |
| 7 | DeepSeek R1 Distill Qwen 14B | 39.28 | #2683 | |
| 8 | Qwen 3 8B | Thinking | 37.86 | #1868 |
| 9 | OlympicCoder-32B | 35.95 | #2519 | |
| 10 | Qwen 3 1.7B | Thinking | 30.24 | #5807 |
| 11 | Qwen 3 0.6B | Thinking | 27.86 | #8337 |
| 12 | DeepSeek-R1-Distill-Qwen-7B | 26.43 | #7957 | |
| 13 | DeepSeek R1 Distill Llama 8B | 22.14 | #6937 | |
| 14 | s1-32B | 20.95 | #1728 | |
| 15 | L1-Qwen-1.5B-Exact | 19.76 |
No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.
Interactive version: theaggregate.ai/benchmark?slug=mathif · How It Works · Data refreshed daily, snapshot 2026-11-17.