SommBench - Wine Feature Completion: leaderboard
Metric: Wine feature completion accuracy (%), mean over eight output languages: attribute accuracy on SommBench's 1,000 wine profiles with one to three masked attributes (type, country, region, grapes, dryness, body, acidity, sugar, alcohol) completed as structured JSON in the target language, numeric attributes counted correct within 5% error, zero-shot, temperature 0, one run; higher is better. Source: arxiv.org. Saturation forecast: Around 2028. 21 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-4o | 63 | #333 |
| 2 | Gemini 2.5 Flash | 63 | #237 |
| 3 | Gemini 2.5 Pro | 62 | #145 |
| 4 | GPT-4.1 | 62 | #240 |
| 5 | GPT-4o Mini | 62 | #588 |
| 6 | GPT-4.1 Mini | 61 | #346 |
| 7 | Grok 4 | 61 | #169 |
| 8 | Grok 4 Fast | 59 | #242 |
| 9 | GPT-5 | 57 | #91 |
| 10 | Gemini 2.5 Flash Lite | 57 | #413 |
| 11 | GPT-4.1 Nano | 52 | #716 |
| 12 | GPT-OSS-120B (Low) | 41 | #330 (GPT-OSS-120B) |
| 13 | GPT-OSS-120B (Medium) | 40 | #330 (GPT-OSS-120B) |
| 14 | GPT-OSS-20B (Medium) | 38 | #499 (GPT-OSS-20B) |
| 15 | GPT-OSS-20B (Low) | 33 | #499 (GPT-OSS-20B) |
No result here: #3 Claude Opus 5.5, #5 GPT-6 Astra, #8 Claude Fable 5.1.
Interactive version: theaggregate.ai/benchmark?slug=sommbench-wine-feature-completion · How It Works · Data refreshed daily, snapshot 2026-10-11.