AutoformBench: leaderboard

Meta FAIR benchmark of 341 graduate- and research-level Lean 4 autoformalization tasks across 24 mathematical sources, evaluated in a shared network-disabled Harbor agent loop with Lean-kernel verification and a $200 per-task cost ceiling.

Metric: Solved (%) of 341 Lean 4 formalization tasks. Source: github.com. Saturation forecast: Around March 2027. 9 models tracked. Updated 20 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-6 AstraMax82.4#10
2Claude Opus 5Max61.88#26
3GPT-5.6 SolxHigh58.94#36
4Claude Fable 5Max53.08#32
5DeepSeek V4.1 FlashMax52.79#69
6Gemini 3.7 FlashHigh52.49#42
7Kimi K3Max48.39#61
8Claude Sonnet 5High44.57#118
9Claude Opus 4.8High43.99#64

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=autoformbench · How It Works · Data refreshed daily, snapshot 2026-11-20.