RFMC - English-Russian: leaderboard
Medical-domain translation from English into Russian on the Reeve Foundation Multilingual Corpus (RFMC; Balashov, VanHorn, Xu and Downes, University of Georgia, EAMT 2026): the 1,143 sentences of Chapter 1 of the Reeve Foundation's Paralysis Resource Guide are translated one call per sentence with a fixed zero-shot prompt and no context, and the whole output is scored with COMET-22 against the foundation's professional Russian translation. The score is document-level COMET-22 x100 (higher is better); local open-weight models run through Ollama, mostly at 4-bit quantization, reasoning models at their default temperature, and failed calls count as empty lines.
Metric: COMET-22 (x100; document-level mean over 1,143 medical-domain sentences, English into Russian, professional reference translations). Source: arxiv.org. Saturation forecast: Estimated already saturated. 16 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.2 | 89.91 | #172 |
| 2 | GPT-OSS-20B | 88.45 | #1150 |
| 3 | Llama 3.1 8B Instruct | 87.01 | #4277 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=rfmc-english-russian · How It Works · Data refreshed daily, snapshot 2026-11-12.