ReuF2F - Pass@2: leaderboard
ReuF2F gives a coding agent one of 244 open problems from undergraduate research programmes, formalised in Lean 4, and asks it to either prove the statement or prove its full negation. Every model runs in the same Leanimum-agent harness inside a container, with up to 1,000 steps and 6 hours per attempt, internet access and two attempts per problem. A submission counts only if the unchanged statement or its negation passes the Comparator checker with both the Lean kernel and nanoda, so there is no LLM judge. Unlike miniF2F, whose statements are olympiad and textbook problems with known proofs, every ReuF2F problem was unsolved when collected, and a run is a multi-hour agent session. The rows are ProbXiv's transcription of the paper's tables (arXiv 2610.27046), including its reasoning-effort sweep. This board is pass@2: the share of the 244 problems proved or refuted at least once in two attempts.
Metric: pass@2: problems proved or refuted at least once in two attempts (%). Source: probxiv.com. Saturation forecast: Around August 2027. 22 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 4 Argon (High) | 44.67 | #11 (Gemini 4 Argon) |
| 2 | Gemini 4 Argon (Medium) | 44.26 | #11 (Gemini 4 Argon) |
| 3 | Claude Opus 5.5 (Max) | 43.85 | #5 (Claude Opus 5.5) |
| 4 | Gemini 4 Argon (Low) | 42.62 | #11 (Gemini 4 Argon) |
| 5 | GPT-6.1 Sol (Max) | 40.98 | #10 (GPT-6.1 Sol) |
| 6 | Claude Opus 5.5 (xHigh) | 39.34 | #5 (Claude Opus 5.5) |
| 7 | GPT-6 Astra (Max) | 37.7 | #6 (GPT-6 Astra) |
| 8 | Claude Opus 5.5 (High) | 35.66 | #5 (Claude Opus 5.5) |
| 9 | Claude Opus 5.5 (Medium) | 28.28 | #5 (Claude Opus 5.5) |
| 10 | GPT-6 Astra (xHigh) | 26.64 | #6 (GPT-6 Astra) |
| 11 | GPT-6.1 Sol (xHigh) | 26.23 | #10 (GPT-6.1 Sol) |
| 12 | GPT-6 Astra (High) | 25.82 | #6 (GPT-6 Astra) |
| 13 | GPT-6.1 Sol (High) | 21.31 | #10 (GPT-6.1 Sol) |
| 14 | GPT-6 Astra (Medium) | 20.9 | #6 (GPT-6 Astra) |
| 15 | GPT-6 Sol (Max) | 19.26 | #24 (GPT-6 Sol) |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon.
Interactive version: theaggregate.ai/benchmark?slug=reuf2f-pass-2 · How It Works · Data refreshed daily, snapshot 2026-11-12.