pythia-2.8B: benchmark results
EleutherAI's Pythia-2.8B, part of the eight-size Pythia Scaling Suite built for interpretability research, trained on the Pile with 154 intermediate checkpoints released for analysis. Provider: EleutherAI. Released 2023-04-03. Access: Open.
Aggregate Rating 1066 ± 33, rank #1650 of 1656 rated models, from 64 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| ToolBench - WebShop Long | 0 | Task Score | 44.2 |
| ToolBench - Google Sheets | 12.9 | Task Score | 41.9 |
| RoBench-25 (Recovery Instruction) - Trigger C | 0.22 | Clipped Heidke skill score (0-1; chance-corrected accuracy o | 40 |
| RoBench-25 (Recovery Instruction) - Trigger E | 0.19 | Clipped Heidke skill score (0-1; chance-corrected accuracy o | 40 |
| ToolBench - VirtualHome | 14.8 | Task Score | 39.5 |
| ToolBench - Tabletop | 7.6 | Task Score | 33.7 |
| RoBench-25 (No Defense Instruction) - Trigger E | 0.14 | Clipped Heidke skill score (0-1; chance-corrected accuracy o | 33.3 |
| RoBench-25 (No Defense Instruction) - Trigger F | 0.05 | Clipped Heidke skill score (0-1; chance-corrected accuracy o | 33.3 |
| ToolBench - Open Weather | 49 | Task Score | 30.2 |
| ToolBench - The Cat API | 54 | Task Score | 25.6 |
| ToolBench Leaderboard | 18.58 | Average Task Score | 25.6 |
| ToolBench - WebShop Short | 0 | Task Score | 24.4 |
Interactive version: theaggregate.ai/model?slug=pythia-2-8b · How It Works · Data refreshed daily, snapshot 2026-10-20.