ContractScrub: leaderboard
Macro F1 for identifying drafting errors and defined-term issues across nine categories in 44 legal contracts. Published reasoning settings and disabled-reasoning comparisons are preserved separately.
Metric: Macro F1 (0-1). Source: arxiv.org. Saturation forecast: Around March 2027. 12 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) (Thinking) | 0.66 | #58 (Gemini 3.1 Pro (Preview)) |
| 2 | Claude Sonnet 4.6 (High) | 0.64 | #90 (Claude Sonnet 4.6) |
| 3 | GPT-5.5 (Medium) | 0.63 | #31 (GPT-5.5) |
| 4 | Claude Opus 4.7 (High) | 0.62 | #49 (Claude Opus 4.7) |
| 5 | Gemini 2.5 Pro (Thinking) | 0.56 | #152 (Gemini 2.5 Pro) |
| 6 | GPT-5.2 (Medium) | 0.55 | #110 (GPT-5.2) |
| 7 | GPT-5.5 (Non-reasoning) | 0.53 | #31 (GPT-5.5) |
| 8 | Claude Haiku 4.5 (High) | 0.49 | #280 (Claude Haiku 4.5) |
| 9 | Claude Opus 4.7 (Non-reasoning) | 0.49 | #49 (Claude Opus 4.7) |
| 10 | O4 Mini | 0.45 | #180 |
| 11 | Gemma 4 26B A4B (Non-reasoning) | 0.33 | #358 (Gemma 4 26B A4B) |
| 12 | Qwen 3.5 397B A17B (Thinking) | 0.32 | #146 (Qwen 3.5 397B A17B) |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=contractscrub · How It Works · Data refreshed daily, snapshot 2026-10-20.