MathAtlas - Definitions: leaderboard
Autoformalization of graduate and research-level mathematical definitions from textbooks and papers into Lean 4, many depending on concepts Mathlib does not yet contain; MathAtlas (Patel et al., 2026). Percent of definitions whose formalization compiles against Mathlib and is judged faithful to the informal text by CriticLean-32B, zero-shot prompting; higher is better.
Metric: Correctness (%; share of all definitions whose Lean 4 formalization compiles against Mathlib (Lean v4.24.0) and is judged faithful to the informal text by CriticLean-32B; zero-shot prompt). Source: arxiv.org. Saturation forecast: Around October 2028. 2 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-OSS-120B | 6.6 | #699 |
| 2 | GPT-OSS-20B | 2.2 | #1150 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=mathatlas-definitions · How It Works · Data refreshed daily, snapshot 2026-11-12.