German Statutory QA - Post-Cutoff Amendments (Vanilla): leaderboard
Metric: Correct outcomes (%; share of 115 expert-validated German statutory questions whose fact date falls after a legislative amendment made after 1 November 2024, past both models' knowledge cutoff whose legal conclusion a Gemini 3 Flash Preview judge scores fully correct, model without retrieval or tools). Source: arxiv.org. Saturation forecast: Around July 2027. 2 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | GPT-5.1 | 40 |
| 2 | Claude 3.7 Sonnet | 24.35 |
Interactive version: theaggregate.ai/benchmark?slug=german-statutory-qa-post-cutoff-amendments-vanilla · How It Works · Data refreshed daily, snapshot 2026-09-26.