mGPT: benchmark results
ai-forever's mGPT, a 1.3B-parameter multilingual GPT-3-style model trained on Wikipedia and the Colossal Clean Crawled Corpus across 61 languages from 25 families. Provider: Sber. Released 2022-04-07. Access: Open.
Aggregate Rating 1046 ± 32, rank #1632 of 1635 rated models, from 31 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| MERA Original Paper - ruModAr | 0.1 | Exact Match (%; greedy generation) | 58.3 |
| MERA Original Paper - ruMultiAr | 1.2 | Exact Match (%; 5-shot, greedy generation) | 58.3 |
| MERA Original Paper - SimpleAr | 0.7 | Exact Match (%; 5-shot, greedy generation) | 55.6 |
| MERA Original Paper | 19.8 | Total Score (%; mean of 17 problem-solving and exam tasks, M | 52.8 |
| MERA Original Paper - MathLogicQA | 25.8 | Accuracy (%; 5-shot, log-likelihood) | 47.2 |
| MERA Original Paper - ruWorldTree | 25.1 | Accuracy (%; 5-shot, log-likelihood) | 47.2 |
| MERA Original Paper - ruMMLU | 24.1 | Accuracy (%; 5-shot, log-likelihood) | 27.8 |
| MERA Original Paper - ruOpenBookQA | 24.5 | Accuracy (%; 5-shot, log-likelihood) | 27.8 |
| EuroEval Danish Knowledge | 4.48 | Knowledge Average Score (%) | 15 |
| EuroEval Icelandic Knowledge | 0.25 | Knowledge Average Score (%) | 12.4 |
| EuroEval Danish Knowledge - Danish Citizen Tests | 4.48 | MCC (x100) | 11.8 |
| Open LLM Leaderboard v1 - TruthfulQA MC2 | 39.62 | MC2 (%) (0-shot) | 11.6 |
Interactive version: theaggregate.ai/model?slug=mgpt · How It Works · Data refreshed daily, snapshot 2026-11-05.