German Statutory QA - Post-Cutoff Amendments (Vanilla): leaderboard
German statutory case questions whose fact date falls after an amendment made past November 2024, so the model must know law newer than its training data (Prior, Schultz and Grabmair, 2026). Share of 115 expert-validated questions whose legal outcome a Gemini 3 Flash Preview judge rates fully correct, in percent, answered without retrieval or web search; higher is better.
Metric: Correct outcomes (%; share of 115 expert-validated German statutory questions whose fact date falls after a legislative amendment made after 1 November 2024, past both models' knowledge cutoff whose legal conclusion a Gemini 3 Flash Preview judge scores fully correct, model without retrieval or tools). Source: arxiv.org. Saturation forecast: Around March 2027. 2 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.1 | 40 | #225 |
| 2 | Claude 3.7 Sonnet | 24.35 | #455 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=german-statutory-qa-post-cutoff-amendments-vanilla · How It Works · Data refreshed daily, snapshot 2026-11-12.