German Statutory QA - Multi-Provision Pre-Amendment (Vanilla): leaderboard
German statutory case questions that combine several provisions, at least one of which applies in an earlier version on the fact date (Prior, Schultz and Grabmair, 2026). Share of 84 expert-validated questions whose legal outcome a Gemini 3 Flash Preview judge rates fully correct, in percent, answered without retrieval or web search; higher is better.
Metric: Correct outcomes (%; share of 84 expert-validated German statutory questions combining several provisions, at least one of which applies in an earlier version whose legal conclusion a Gemini 3 Flash Preview judge scores fully correct, model without retrieval or tools). Source: arxiv.org. Saturation forecast: Around September 2027. 3 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.2 | 42.86 | #172 |
| 2 | DeepSeek V3.2 | 27.38 | #403 |
| 3 | Claude Opus 4.5 | 26.19 | #152 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=german-statutory-qa-multi-provision-pre-amendment-vanilla · How It Works · Data refreshed daily, snapshot 2026-11-12.