MathIF - Correctness with Constraints: leaderboard

The same 420 constrained MathIF questions, one sample each, scored by whether the final answer is correct regardless of whether the instructions were followed.

Metric: Correctness (%): share of the same 420 constrained questions answered correctly, 1 sample per question. Source: raw.githubusercontent.com. Saturation forecast: Estimated already saturated. 23 models tracked. Updated 17 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1Qwen 3 32BThinking70#965
2QwQ-32B68.81#908
3Open-Reasoner-Zero-32B67.62
4Qwen 3 8BThinking66.43#1868
5Qwen 3 14BReasoning64.29#1264
6s1-32B60.95#1728
7Qwen 3 4BReasoning58.57#2337
8DeepSeek R1 Distill Qwen 32B57.62#1806
9OlympicCoder-32B54.52#2519
10DeepSeek R1 Distill Llama 70B54.05#1669
11Open-Reasoner-Zero-7B51.9
12Qwen 3 1.7BThinking51.19#5807
13DeepSeek R1 Distill Qwen 14B50.95#2683
14DeepSeek-R1-Distill-Qwen-7B48.57#7957
15L1-Qwen-1.5B-Max45.71

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=mathif-correctness-with-constraints · How It Works · Data refreshed daily, snapshot 2026-11-17.