pythia-410m: benchmark results
EleutherAI's Pythia-410M, part of the eight-size Pythia Scaling Suite built for interpretability research, trained on the Pile with 154 intermediate checkpoints released for analysis. Provider: EleutherAI. Released 2023-02-13. Access: Open.
Aggregate Rating 1105 ± 30, rank #1622 of 1635 rated models, from 17 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Portuguese LLM - BLUEX | 23.64 | Accuracy (%) | 17.3 |
| Open Portuguese LLM - FaQuAD NLI | 43.97 | Macro F1 (%) | 17.1 |
| Open LLM Leaderboard - MuSR | 35.78 | Score | 16.7 |
| Open LLM Leaderboard v1 - TruthfulQA MC2 | 41.22 | MC2 (%) (0-shot) | 16.4 |
| Open LLM Leaderboard v1 - MMLU | 27.25 | Accuracy (%) (5-shot) | 14.9 |
| Open LLM Leaderboard - IFEval | 21.95 | Score | 14.8 |
| Open LLM Leaderboard v1 - GSM8K | 0.68 | Accuracy (%) (5-shot) | 13.5 |
| Open Portuguese LLM - OAB Exams | 24.56 | Accuracy (%) | 13.4 |
| Open LLM Leaderboard - GPQA | 25.92 | Score | 11.7 |
| Open Portuguese LLM - ENEM | 19.8 | Accuracy (%) | 11.6 |
| Open LLM Leaderboard v1 - HellaSwag | 40.85 | Normalized accuracy (%) (10-shot) | 8.2 |
| Open LLM Leaderboard v1 - WinoGrande | 53.12 | Accuracy (%) (5-shot) | 7.3 |
Interactive version: theaggregate.ai/model?slug=pythia-410m · How It Works · Data refreshed daily, snapshot 2026-11-05.