pythia-12B-sft-v8-7k-steps: benchmark results
Provider: EleutherAI. Access: Open.
Aggregate Rating 1153 ± 16, rank #9932 of 10916 rated models, from 68 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| MMLU-by-task - Abstract Algebra | 31 | Accuracy (%) | 60.2 |
| MMLU-by-task - Machine Learning | 31.25 | Accuracy (%) | 43.7 |
| MMLU-by-task - College Chemistry | 29 | Accuracy (%) | 38 |
| MMLU-by-task - High School Statistics | 32.41 | Accuracy (%) | 36.7 |
| MMLU-by-task - ARC Challenge | 40.53 | Accuracy (%) | 34.7 |
| MMLU-by-task - Jurisprudence | 34.26 | Accuracy (%) | 34.7 |
| MMLU-by-task - HellaSwag | 52.04 | Accuracy (%) | 34.2 |
| MMLU-by-task - High School US History | 29.9 | Accuracy (%) | 33.8 |
| MMLU-by-task - Human Sexuality | 32.06 | Accuracy (%) | 33.8 |
| MMLU-by-task - World Religions | 35.09 | Accuracy (%) | 33.6 |
| MMLU-by-task - Medical Genetics | 34 | Accuracy (%) | 33.2 |
| MMLU-by-task - Security Studies | 35.51 | Accuracy (%) | 32.8 |
Interactive version: theaggregate.ai/model?slug=pythia-12b-sft-v8-7k-steps · How It Works · Data refreshed daily, snapshot 2026-11-08.