pythia-1B: benchmark results
Provider: EleutherAI. Released 2023-04-03. Access: Open.
Unified ELO 1184 ± 25, rank #2642 of 2656 rated models, from 23 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Portuguese LLM - FaQuAD NLI | 43.97 | Macro F1 (%) | 17.1 |
| Open LLM Leaderboard - MuSR | 35.52 | Score | 15.2 |
| Open LLM Leaderboard - IFEval | 22.08 | Score | 15.1 |
| Open Portuguese LLM - ASSIN2 RTE | 46.5 | Macro F1 (%) | 14.8 |
| Open Portuguese LLM - OAB Exams | 24.15 | Accuracy (%) | 12.1 |
| Open Medical LLM - MMLU Professional Medicine | 26.84 | Accuracy (%) | 11.9 |
| HELM Classic - bAbI | 37.86 | Exact Match (%) | 11.6 |
| Open Medical LLM - MedMCQA | 31.63 | Accuracy (%) | 10.8 |
| Open Medical LLM - PubMedQA | 58.2 | Accuracy (%) | 10.8 |
| Open LLM Leaderboard - GPQA | 25.67 | Score | 9 |
| Open Medical LLM - MMLU College Biology | 27.08 | Accuracy (%) | 8.5 |
| Open Portuguese LLM - BLUEX | 19.19 | Accuracy (%) | 8.1 |
Interactive version: theaggregate.ai/model?slug=pythia-1b · How It Works · Data refreshed daily, snapshot 2026-09-19.