ruGPT3-large: benchmark results
ai-forever's ruGPT3-large, a Russian GPT-2-style model trained by SberDevices on 80B tokens, then extended to a 2048-token context. Provider: Sber. Released 2020-10-22. Access: Open.
Aggregate Rating 1129 ± 15, rank #10217 of 10870 rated models, from 99 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| MMLU-by-task - High School Statistics | 47.22 | Accuracy (%) | 83.3 |
| MMLU-by-task - High School Mathematics | 28.89 | Accuracy (%) | 67.9 |
| MERA Original Paper - ruModAr | 0.1 | Exact Match (%; greedy generation) | 58.3 |
| MERA Original Paper - SimpleAr | 0.4 | Exact Match (%; 5-shot, greedy generation) | 50 |
| MMLU-by-task - Professional Medicine | 45.22 | Accuracy (%) | 48.2 |
| MERA - LCS | 12 | Accuracy (%) | 44.5 |
| MERA Original Paper - ruMultiAr | 0.7 | Exact Match (%; 5-shot, greedy generation) | 44.4 |
| MMLU-by-task - High School Chemistry | 31.03 | Accuracy (%) | 43.8 |
| MMLU-by-task - TruthfulQA MC2 | 43.39 | Accuracy (%) | 42.8 |
| MMLU-by-task - College Physics | 22.55 | Accuracy (%) | 34.2 |
| MMLU-by-task - Abstract Algebra | 27 | Accuracy (%) | 34.1 |
| MERA Original Paper - ruMMLU | 24.5 | Accuracy (%; 5-shot, log-likelihood) | 33.3 |
Interactive version: theaggregate.ai/model?slug=rugpt3-large · How It Works · Data refreshed daily, snapshot 2026-11-12.