GaelEval - English to Gaelic Translation (chrF): leaderboard

GaelEval (2026) is a Scottish Gaelic benchmark from the DR-LIB project. This board is chrF, a character n-gram F-score (0 to 100), for translating 908 An Litir Bheag podcast transcripts from English into Gaelic, scored against the professional parallel translations; higher is better.

Metric: chrF character n-gram F-score of English-to-Gaelic translation against gold references; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 17 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 3 Flash (Preview)79.07#156
2Gemini 3 Pro (Preview)78.57#123
3Gemini 2.5 Flash74.72#466
4GPT-4.174.62#506
5GPT-5.272.16#190
6DeepSeek R171.84#484
7GPT-570.27#185
8Claude Opus 4.669.34#114
9GPT-4o67.83#754
10Llama 4 Maverick62.3#1113
11GPT-5 Mini60.93#373
12Claude Haiku 4.559.27#587
13GPT-4.1 Mini58.01#799
14GPT-4o Mini56.84#1680
15GPT-5 Nano55.54#912

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=gaeleval-english-to-gaelic-translation-chrf · How It Works · Data refreshed daily, snapshot 2026-11-22.