pythia-1B: benchmark results
EleutherAI's Pythia-1B, part of the eight-size Pythia Scaling Suite built for interpretability research, trained on the Pile with 154 intermediate checkpoints released for analysis. Provider: EleutherAI. Released 2023-04-03. Access: Open.
Aggregate Rating 1142 ± 34, rank #1608 of 1635 rated models, from 23 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Portuguese LLM - FaQuAD NLI | 43.97 | Macro F1 (%) | 17.1 |
| Open LLM Leaderboard - MuSR | 35.52 | Score | 15.2 |
| Open LLM Leaderboard - IFEval | 22.08 | Score | 15.1 |
| Open Portuguese LLM - ASSIN2 RTE | 46.5 | Macro F1 (%) | 14.7 |
| Open Portuguese LLM - OAB Exams | 24.15 | Accuracy (%) | 12.2 |
| HELM Classic - bAbI | 37.86 | Exact Match (%) | 11.6 |
| Open Medical LLM - MMLU Professional Medicine | 26.84 | Accuracy (%) | 11.5 |
| Open Medical LLM - MedMCQA | 31.63 | Accuracy (%) | 10.4 |
| Open Medical LLM - PubMedQA | 58.2 | Accuracy (%) | 9.9 |
| Open LLM Leaderboard - GPQA | 25.67 | Score | 9 |
| Open Portuguese LLM - BLUEX | 19.19 | Accuracy (%) | 8.2 |
| Open Medical LLM - MMLU College Biology | 27.08 | Accuracy (%) | 8.1 |
Interactive version: theaggregate.ai/model?slug=pythia-1b · How It Works · Data refreshed daily, snapshot 2026-11-05.