T2 Theorem Testing: leaderboard

T2 (Theorem Testing, 2026) asks a model to write a Lean 4 theorem from its natural-language statement and repository context, then tests it like code: the theorem passes only if every successor theorem that depends on it still compiles. Testing accuracy (%) over 2,206 problems from five real-world Lean repositories; higher is better.

Metric: Testing accuracy (%): share of T2's 2,206 problems from five Lean 4 repositories in which the generated theorem, substituted for the original, lets every dependent successor theorem compile; zero-shot from a natural-language statement (written by Claude Sonnet 4.5) with predecessor and successor context, one completion at temperature 0.6, top-p 0.95; higher is better. Source: arxiv.org. Saturation forecast: Around 2030. 18 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Claude Sonnet 4.538.9#261
2GPT-5 Mini37.9#364
3GPT-537.7#178
4Llama 3.1 70B37#1499
5Claude 3.7 Sonnet36.8#470
6GPT-4o Mini36.7#1667
7GPT-5 Nano36.6#906
8DeepSeek R136.3#482
9Claude Sonnet 436#410
10Llama 3.1 405B33.2#1082
11GPT-OSS-120B32.3#704
12Goedel-Prover-V2-32B31
13DeepSeek-Prover-V2-7B30
14Llama 3.1 8B24.8#5196
15Goedel-Prover-V2-8B24.5

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=t2-theorem-testing · How It Works · Data refreshed daily, snapshot 2026-11-21.