TRACK - Math Reasoning (Facts in Context, Thinking): leaderboard
Knowledge-conflict reasoning in TRACK (Testing Reasoning Amid Conflicting Knowledge; Feng et al., EACL 2026): 500 PRM800K-derived problems decomposed into procedural steps, where the needed facts contradict or extend what the model learned in pre-training, answered with the missing facts appended to the prompt and longer thinking enabled. Holistic pass in percent (a correct final answer and reasoning that entails every required fact), higher is better.
Metric: Holistic pass (%; PRM800K-derived problems decomposed into procedural steps, 500 items; the final answer is correct and the reasoning entails every required fact; answered with the missing facts appended to the prompt and longer thinking enabled). Source: arxiv.org. Saturation forecast: Estimated already saturated. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | O4 Mini (2025-04-16) | 87.2 | #362 |
| 2 | Qwen 3 8B (Thinking) | 70.7 | #1838 (Qwen 3 8B) |
| 3 | Qwen 3 4B (Reasoning) | 36.5 | #2297 (Qwen 3 4B) |
| 4 | Qwen 3 1.7B (Thinking) | 7.1 | #5551 (Qwen 3 1.7B) |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=track-math-reasoning-facts-in-context-thinking · How It Works · Data refreshed daily, snapshot 2026-11-12.