ruGPT-3-small: benchmark results
ai-forever's ruGPT-3-small, a Russian GPT-2-style model trained by SberDevices on 80B tokens, then extended to a 2048-token context. Provider: Sber. Released 2020-10-28. Access: Open.
Aggregate Rating 1189 ± 34, rank #9600 of 10870 rated models, from 31 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| MERA Original Paper - ruMMLU | 26.3 | Accuracy (%; 5-shot, log-likelihood) | 61.1 |
| MERA Original Paper - ruWorldTree | 25.7 | Accuracy (%; 5-shot, log-likelihood) | 61.1 |
| MERA Original Paper - ruModAr | 0.1 | Exact Match (%; greedy generation) | 58.3 |
| MERA Original Paper - ruMultiAr | 0.9 | Exact Match (%; 5-shot, greedy generation) | 50 |
| MERA Original Paper - ruOpenBookQA | 25.8 | Accuracy (%; 5-shot, log-likelihood) | 50 |
| MERA - LCS | 10.4 | Accuracy (%) | 29 |
| MERA Original Paper - SimpleAr | 0 | Exact Match (%; 5-shot, greedy generation) | 22.2 |
| MERA - USE | 8.14 | Grade, normalized (%) | 21.7 |
| MERA - ruHateSpeech | 54.34 | Accuracy (%) | 13.6 |
| MERA Original Paper - MathLogicQA | 24.4 | Accuracy (%; 5-shot, log-likelihood) | 11.1 |
| MERA - RWSD | 46.15 | Accuracy (%) | 10.5 |
| MERA Original Paper | 19.1 | Total Score (%; mean of 17 problem-solving and exam tasks, M | 8.3 |
Interactive version: theaggregate.ai/model?slug=rugpt-3-small · How It Works · Data refreshed daily, snapshot 2026-11-12.