PatRe - Office Action Quality (OA-RO): leaderboard
PatRe (2026) models patent examination on 480 recent USPTO cases across all eight IPC sections. Acting as the examiner, the model writes an office action from the claims together with the references the examiner and applicant cited (reference oracle). A Gemini 3.1 Flash Lite judge rates soundness, clarity, constructiveness, completeness and legal style from 1 to 10, and the score is their mean; higher is better.
Metric: Overall office action quality on a 1-10 scale: mean of the soundness, clarity, constructiveness, completeness and legal-style ratings of a Gemini-3.1-Flash-Lite judge at temperature 0, office action generation under reference oracle: the model also receives the references the examiner and applicant cited and must pick the relevant ones, over PatRe (480 recent USPTO patent examination histories across all eight IPC sections, with office actions, rebuttals, claim versions and cited references), temperature 0; higher is better. Source: arxiv.org. Saturation forecast: Around December 2027. 10 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5 Mini | 4.89 | #364 |
| 2 | Qwen 3.5 27B | 4.37 | #375 |
| 3 | Gemini 2.5 Flash | 4.36 | #463 |
| 4 | DeepSeek V3.2 | 4.34 | #419 |
| 5 | Qwen 3.5 9B | 4.11 | #828 |
| 6 | Gemma 3 27B (IT) | 3.65 | #1324 |
| 7 | GPT-4o Mini | 3.61 | #1667 |
| 8 | Gemma 3 12B (IT) | 3.59 | #1887 |
| 9 | Llama 3.3 70B Instruct | 3.4 | #1385 |
| 10 | Llama 3.1 8B Instruct | 3.07 | #4600 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=patre-office-action-quality-oa-ro · How It Works · Data refreshed daily, snapshot 2026-11-21.