PatRe - Office Action Quality (OA-RS): leaderboard

PatRe (2026) models patent examination on 480 recent USPTO cases across all eight IPC sections. Acting as the examiner, the model writes an office action from the claims together with a noisy pool of retrieved references mixed with the cited ones (retrieval simulated). A Gemini 3.1 Flash Lite judge rates soundness, clarity, constructiveness, completeness and legal style from 1 to 10, and the score is their mean; higher is better.

Metric: Overall office action quality on a 1-10 scale: mean of the soundness, clarity, constructiveness, completeness and legal-style ratings of a Gemini-3.1-Flash-Lite judge at temperature 0, office action generation under retrieval simulated: the model receives a noisy pool of BM25-retrieved references mixed with ground-truth ones, over PatRe (480 recent USPTO patent examination histories across all eight IPC sections, with office actions, rebuttals, claim versions and cited references), temperature 0; higher is better. Source: arxiv.org. Saturation forecast: Around August 2027. 9 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1GPT-5 Mini5.39#364
2Gemini 2.5 Flash4.37#463
3Qwen 3.5 27B4.3#375
4Qwen 3.5 9B4.11#828
5Gemma 3 27B (IT)3.68#1324
6GPT-4o Mini3.59#1667
7Gemma 3 12B (IT)3.51#1887
8Llama 3.3 70B Instruct3.5#1385
9Llama 3.1 8B Instruct3.19#4600

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=patre-office-action-quality-oa-rs · How It Works · Data refreshed daily, snapshot 2026-11-21.