MathIF - Correctness with Constraints: leaderboard
The same 420 constrained MathIF questions, one sample each, scored by whether the final answer is correct regardless of whether the instructions were followed.
Metric: Correctness (%): share of the same 420 constrained questions answered correctly, 1 sample per question. Source: raw.githubusercontent.com. Saturation forecast: Estimated already saturated. 23 models tracked. Updated 17 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Qwen 3 32B | Thinking | 70 | #965 |
| 2 | QwQ-32B | 68.81 | #908 | |
| 3 | Open-Reasoner-Zero-32B | 67.62 | ||
| 4 | Qwen 3 8B | Thinking | 66.43 | #1868 |
| 5 | Qwen 3 14B | Reasoning | 64.29 | #1264 |
| 6 | s1-32B | 60.95 | #1728 | |
| 7 | Qwen 3 4B | Reasoning | 58.57 | #2337 |
| 8 | DeepSeek R1 Distill Qwen 32B | 57.62 | #1806 | |
| 9 | OlympicCoder-32B | 54.52 | #2519 | |
| 10 | DeepSeek R1 Distill Llama 70B | 54.05 | #1669 | |
| 11 | Open-Reasoner-Zero-7B | 51.9 | ||
| 12 | Qwen 3 1.7B | Thinking | 51.19 | #5807 |
| 13 | DeepSeek R1 Distill Qwen 14B | 50.95 | #2683 | |
| 14 | DeepSeek-R1-Distill-Qwen-7B | 48.57 | #7957 | |
| 15 | L1-Qwen-1.5B-Max | 45.71 |
No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.
Interactive version: theaggregate.ai/benchmark?slug=mathif-correctness-with-constraints · How It Works · Data refreshed daily, snapshot 2026-11-17.