GaelEval - English to Gaelic Translation (BLEU): leaderboard

GaelEval (2026) is a Scottish Gaelic benchmark from the DR-LIB project. This board is corpus BLEU (0 to 100) for translating 908 An Litir Bheag podcast transcripts from English into Gaelic, scored against the professional parallel translations; higher is better.

Metric: BLEU-4 of English-to-Gaelic translation of An Litir Bheag transcripts against hand-translated gold references; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 17 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 3 Flash (Preview)71.47#152
2Gemini 3 Pro (Preview)65.56#119
3Gemini 2.5 Flash65.53#463
4GPT-4.165.41#505
5DeepSeek R162.12#482
6GPT-5.261.19#185
7GPT-556.94#178
8Claude Opus 4.653.16#113
9GPT-4o52.39#751
10GPT-5 Mini49.19#364
11Llama 4 Maverick42.93#1110
12GPT-5 Nano42.56#906
13GPT-4.1 Mini38.9#789
14Claude Haiku 4.538.44#583
15GPT-OSS-120B37.79#704

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=gaeleval-english-to-gaelic-translation-bleu · How It Works · Data refreshed daily, snapshot 2026-11-21.