German LLM Index - GermEval NER: leaderboard
German Language LLM Index (dach.peerbench.ai, a PeerBench project, 2026), which reruns every model itself with lm-evaluation-harness: chat template, temperature 0, up to 8,192 generated tokens, thinking models set to minimal reasoning effort where the API allows it and other models run as served. Labels carry (Reasoning off) when the site classed the run as reasoning-off (at most ten reasoning tokens, or under 0.1% of its completion tokens); other rows reasoned at the provider default or at the effort the label names. Native German named-entity recognition (GermEval 2014 sentences): the model marks persons, locations, organisations and miscellaneous entities and returns them as JSON, 1,024 items scored with seqeval micro-F1 over the person, location and organisation classes (the noisy MISC class is excluded). The sentences come from the EuroEval GermEval split, but the run uses lm-evaluation-harness with the prompt of the site rather than the EuroEval harness, so it is kept apart from the EuroEval German GermEval board. Higher is better. Source: https://dach.peerbench.ai/
Metric: seqeval micro-F1 excluding MISC (%). Source: dach.peerbench.ai. Saturation forecast: Around February 2027. 45 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Fable 5 | 87.74 | #20 |
| 2 | Muse Spark Open | 87.66 | #42 |
| 3 | Gemini 3.1 Pro (Preview) | 87.33 | #64 |
| 4 | Gemini 3.8 Flash | 87.2 | #30 |
| 5 | Gemini 3.7 Flash | 86.92 | #24 |
| 6 | GLM-5.3 Flash | 86.42 | #77 |
| 7 | GPT-5.5 (Low) | 85.93 | #33 (GPT-5.5) |
| 8 | Claude Opus 4.8 (Non-reasoning) | 85.84 | #39 (Claude Opus 4.8) |
| 9 | Gemini 3 Flash (Preview) (Non-reasoning) | 84.73 | #90 (Gemini 3 Flash (Preview)) |
| 10 | GPT-5.6 Terra (Non-reasoning) | 83.26 | #40 (GPT-5.6 Terra) |
| 11 | Gemini 3.1 Flash Lite (Non-reasoning) | 83.24 | #200 (Gemini 3.1 Flash Lite) |
| 12 | Gemma 4 31B (IT) (Non-reasoning) | 82.64 | #178 (Gemma 4 31B (IT)) |
| 13 | DeepSeek V4 Pro (Non-reasoning) | 82.21 | #112 (DeepSeek V4 Pro) |
| 14 | Qwen 3.7 Max (Non-reasoning) | 82.07 | #85 (Qwen 3.7 Max) |
| 15 | GPT-5.6 Luna (Non-reasoning) | 81.9 | #79 (GPT-5.6 Luna) |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=german-llm-index-germeval-ner · How It Works · Data refreshed daily, snapshot 2026-11-05.