German LLM Index - INCLUDE (139 questions): leaderboard
German Language LLM Index (dach.peerbench.ai, a PeerBench project, 2026), which reruns every model itself with lm-evaluation-harness: chat template, temperature 0, up to 8,192 generated tokens, thinking models set to minimal reasoning effort where the API allows it and other models run as served. Labels carry (Reasoning off) when the site classed the run as reasoning-off (at most ten reasoning tokens, or under 0.1% of its completion tokens); other rows reasoned at the provider default or at the effort the label names. German part of INCLUDE (CohereLabs include-base-44): native German exam and licensing questions on regional knowledge such as history, law and civics, written in German rather than translated, four options. This board holds runs on all 139 source questions, including the 32 a later blinded audit found duplicated, wrongly keyed, non-unique, corrupted or underspecified; exact match after the custom-extract answer filter. Higher is better. Source: https://dach.peerbench.ai/
Metric: Exact match (%). Source: dach.peerbench.ai. Saturation forecast: Around February 2028. 28 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) | 77.7 | #64 |
| 2 | GPT-5.5 (Low) | 74.82 | #33 (GPT-5.5) |
| 3 | MiniMax-M2.7 | 72.66 | #270 |
| 4 | Claude Fable 5 | 72.66 | #20 |
| 5 | Claude Sonnet 5 (Non-reasoning) | 72.66 | #65 (Claude Sonnet 5) |
| 6 | Gemini 3 Flash (Preview) (Non-reasoning) | 71.94 | #90 (Gemini 3 Flash (Preview)) |
| 7 | Claude Opus 4.8 (Non-reasoning) | 71.94 | #39 (Claude Opus 4.8) |
| 8 | Gemini 3.1 Flash Lite (Non-reasoning) | 71.94 | #200 (Gemini 3.1 Flash Lite) |
| 9 | Qwen 3.7 Max (Non-reasoning) | 71.22 | #85 (Qwen 3.7 Max) |
| 10 | Gemini 2.5 Flash (Non-reasoning) | 70.5 | #255 (Gemini 2.5 Flash) |
| 11 | DeepSeek V4 Flash (Non-reasoning) | 70.5 | #155 (DeepSeek V4 Flash) |
| 12 | DeepSeek V4 Pro (Non-reasoning) | 70.5 | #112 (DeepSeek V4 Pro) |
| 13 | Kimi K2.6 (Non-reasoning) | 69.78 | #113 (Kimi K2.6) |
| 14 | Qwen 3.6 35B A3B (Non-reasoning) | 68.35 | #238 (Qwen 3.6 35B A3B) |
| 15 | Gemma 4 31B (IT) (Non-reasoning) | 68.35 | #178 (Gemma 4 31B (IT)) |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=german-llm-index-include-139-questions · How It Works · Data refreshed daily, snapshot 2026-11-05.