SwitzerlandQA: leaderboard

Each instruction-tuned model answers four-option multiple-choice questions on Swiss geography, history, social life, politics and insurance at national, cantonal and commune level, asked in German, French, Italian, Romansh and English, 0-shot with its chat template; the score is the exact-match percentage. Unlike multilingual cultural sets such as INCLUDE or BLEnD, it covers one country down to individual communes. The Apertus team built it in 2025 and used it while developing Apertus, so Apertus's own scores are not shown; the report does not state how many of the 9,167 items per language each model was scored on.

Metric: Exact match (%). Source: arxiv.org. Saturation forecast: Around February 2027. 18 models tracked. Updated 17 Nov 2026.

Top models

#ModelScoreOverall rank
1Llama 3.3 70B Instruct74.3#1387
2Gemma 3 27B (IT)71#1339
3Qwen 2.5 72B Instruct70#1119
4Qwen 3 32B65.6#965
5Gemma 3 12B (IT)65.1#1891
6OLMo-2-0325-32B-Instruct62.2#2825
7K2-Chat62#3424
8EuroLLM-22B-Instruct-Preview61.6#2763
9EuroLLM-9B-Instruct61.2#3701
10Qwen 3 8B60.7#1868
11Llama 3.1 8B Instruct60#4576
12salamandra-7B-instruct59.6#7499
13SmolLM3-3B56.6#6533
14ALLaM-7B-Instruct-preview56#1989
15Marin 8B Instruct53.7#7507

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=switzerlandqa · How It Works · Data refreshed daily, snapshot 2026-11-17.