MOEVE - Question Answering: leaderboard
MOEVE (Modelle fuer die Oeffentliche Verwaltung Evaluieren, Bundesdruckerei, 2026) benchmarks language models for the German public sector with German prompts and German datasets, many of them from public administration. This board is the mean overall question-answering score on a 0-100 scale over three German datasets, combining embedding similarity with LLM-judged correctness, faithfulness and noise sensitivity; higher is better.
Metric: Mean overall question-answering score x 100 (0-100) over three German datasets (German-QuAD, KIKC QA, FAQ Law), combining embedding similarity with LLM-judged factual correctness, faithfulness and noise sensitivity; German prompts, default decoding, open-weight models on vLLM or Ollama at the quantization listed in the paper's model table, others by API; the paper prints only the top 10 of 39 evaluated models; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 10 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Mistral Large 2 (Nov) Instruct (2411) | 70.4 | #1198 |
| 2 | GPT-OSS-120B | 70.3 | #700 |
| 3 | GPT-4o | 69.2 | #741 |
| 4 | Llama 3.3 70B | 69.1 | #1498 |
| 5 | DeepSeek R1 | 68.9 | #473 |
| 6 | Phi-4 | 68.9 | #1913 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=moeve-question-answering · How It Works · Data refreshed daily, snapshot 2026-11-13.