ruGPT-3-medium: benchmark results
Provider: Sber. Released 2020-10-22. Access: Open.
Aggregate Rating 1170 ± 35, rank #9690 of 10870 rated models, from 31 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| MERA Original Paper - ruOpenBookQA | 27.3 | Accuracy (%; 5-shot, log-likelihood) | 72.2 |
| MERA Original Paper - ruMMLU | 27.1 | Accuracy (%; 5-shot, log-likelihood) | 66.7 |
| MERA Original Paper | 20.1 | Total Score (%; mean of 17 problem-solving and exam tasks, M | 63.9 |
| MERA Original Paper - SimpleAr | 0.8 | Exact Match (%; 5-shot, greedy generation) | 61.1 |
| MERA Original Paper - ruModAr | 0.1 | Exact Match (%; greedy generation) | 58.3 |
| MERA Original Paper - ruMultiAr | 1.2 | Exact Match (%; 5-shot, greedy generation) | 58.3 |
| MERA Original Paper - ruWorldTree | 25.1 | Accuracy (%; 5-shot, log-likelihood) | 47.2 |
| MERA Original Paper - MathLogicQA | 24.8 | Accuracy (%; 5-shot, log-likelihood) | 22.2 |
| MERA - CheGeKa | 1.56 | F1 (%) | 17.1 |
| MERA - USE | 6.96 | Grade, normalized (%) | 15.5 |
| MERA - LCS | 7.4 | Accuracy (%) | 11.2 |
| MERA - ruHumanEval | 0.12 | pass@1 (%) | 10 |
Interactive version: theaggregate.ai/model?slug=rugpt-3-medium · How It Works · Data refreshed daily, snapshot 2026-11-12.