MathArena - ARXIV_FALSE September: leaderboard
Normalized 0–3 rubric score on 53 plausible but false mathematical statements from September 2026 arXiv papers. Full credit requires explicitly identifying the statement as false; a directly contradictory proposed correction caps the score at1. The judge classifies response behavior, not proof validity. September harness: coding tools without internet, 12-hour and $100 limit per attempt. Published percentage, with source model and effort settings retained; separate from final-answer and Lean proof scores. Source: https://matharena.ai/?comp=arxiv_false--september
Metric: Accuracy (%). Source: matharena.ai. Saturation forecast: Around January 2027. 7 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-6.1 Sol (Max) | 82.7 | #8 (GPT-6.1 Sol) |
| 2 | Claude Opus 5.5 (High) | 76.73 | #3 (Claude Opus 5.5) |
| 3 | DeepSeek V4.1 Flash (Max) | 54.72 | #35 (DeepSeek V4.1 Flash) |
| 4 | Gemini 4 Argon | 49.69 | #7 |
| 5 | Qwen3.8 2.4T A95B | 33.02 | #55 |
| 6 | Grok 4.7 (xHigh) | 27.67 | #26 (Grok 4.7) |
| 7 | Gemini 3.8 Flash | 26.73 | #21 |
No result here: #5 GPT-6 Astra, #9 Claude Fable 5.1, #10 Claude Sonnet 5.5.
Interactive version: theaggregate.ai/benchmark?slug=matharena-arxiv-false-september · How It Works · Data refreshed daily, snapshot 2026-10-17.