ruT5-base: benchmark results
ai-forever's ruT5-base, a 222M-parameter Russian encoder-decoder T5 model trained by SberDevices on 300GB of text. Provider: Sber. Released 2021-07-06. Access: Open.
Aggregate Rating 1169 ± 37, rank #9694 of 10870 rated models, from 31 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| MERA Original Paper - ruOpenBookQA | 26.5 | Accuracy (%; 5-shot, log-likelihood) | 63.9 |
| MERA Original Paper - MathLogicQA | 25.9 | Accuracy (%; 5-shot, log-likelihood) | 55.6 |
| MERA Original Paper - SimpleAr | 0 | Exact Match (%; 5-shot, greedy generation) | 22.2 |
| MERA Original Paper - ruWorldTree | 23.4 | Accuracy (%; 5-shot, log-likelihood) | 22.2 |
| MERA Original Paper | 19.3 | Total Score (%; mean of 17 problem-solving and exam tasks, M | 19.4 |
| MERA Original Paper - ruModAr | 0 | Exact Match (%; greedy generation) | 19.4 |
| MERA Original Paper - ruMultiAr | 0 | Exact Match (%; 5-shot, greedy generation) | 19.4 |
| MERA - LCS | 8.8 | Accuracy (%) | 18.1 |
| MERA Original Paper - ruMMLU | 23.7 | Accuracy (%; 5-shot, log-likelihood) | 16.7 |
| MERA - ruHHH | 49.44 | Accuracy (%) | 14.1 |
| MERA - RWSD | 47.31 | Accuracy (%) | 13.1 |
| MERA - RCB | 36.99 | Accuracy (%) | 11.2 |
Interactive version: theaggregate.ai/model?slug=rut5-base · How It Works · Data refreshed daily, snapshot 2026-11-12.