The Shared Word (TSW-NL): leaderboard

Dutch word puzzles from the University of Groningen: three clues each point to one Dutch word, and the model must name that word with no options to choose from, so the answer depends on Dutch-specific lexical relations that do not survive translation. 200 human-curated test items, zero-shot and closed-book, scored by exact match against reviewed accepted answers and averaged over all six clue orders; nine human annotators, with three answers per item, averaged 54.0%.

Metric: Test accuracy (%), mean over six clue orders. Source: arxiv.org. Saturation forecast: Around February 2027. 23 models tracked. Updated 19 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-5.6 SolMax88.8#36
2GPT-5.6 LunaMax79#133
3GPT-4.1 (2025-04-14)59.2#506
4GPT-4o (2024-11-20)56.3#860
5Gemma 3 27B (IT)39.3#1339
6aya-expanse-32B32.9#1971
7Ministral-3-14B-Instruct-251230.8#1646
8Gemma 3 12B (IT)29.8#1893
9Qwen 3 8B28.5#1872
10Qwen 3.5 9B19.9#829
11Llama 3.1 8B Instruct15.8#4591
12Gemma 3 4B (IT)15.6#4146
13ChocoLlama 3 8B Instruct15
14GEITje-7B-ultra14.8#8174
15Llama 3 8B Instruct14.2#4849

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=the-shared-word-tsw-nl · How It Works · Data refreshed daily, snapshot 2026-11-20.