MathArena - BrokenArXiv March (Allow Disproving): leaderboard

A variant of the MathArena BrokenArXiv benchmark (2026) in which models may disprove: for each of the 56 March 2026 problems the model judges both the original true statement from a recent arXiv paper and its perturbed false version. Scored as binary true-or-false accuracy in percent over the 112 statements, where 50 is chance; higher is better.

Metric: Binary classification accuracy (%) over 112 statements, the original true statement and its perturbed false version for each of the 56 problems of the March 2026 BrokenArXiv set (plausible statements from recent arXiv papers perturbed to be false), MathArena's provider-recommended settings, no tools, with a prompt that allows disproving and asks whether each statement is true or false (chance 50); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 3 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 3.1 Pro (Preview)71#99
2MathArena GLM 5.1 (checkpoint unspecified)56.3
3Step 3.5 Flash46.7#526

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=matharena-brokenarxiv-march-allow-disproving · How It Works · Data refreshed daily, snapshot 2026-11-21.