pythia-1.4B: benchmark results
EleutherAI's Pythia-1.4B, part of the eight-size Pythia Scaling Suite built for interpretability research, trained on the Pile with 154 intermediate checkpoints released for analysis. Provider: EleutherAI. Released 2023-04-03. Access: Open.
Aggregate Rating 1150 ± 23, rank #1626 of 1656 rated models, from 33 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| ToolBench - WebShop Short | 5.2 | Task Score | 69.8 |
| ToolBench - WebShop Long | 0 | Task Score | 44.2 |
| ToolBench - Google Sheets | 10 | Task Score | 34.9 |
| ToolBench - Tabletop | 7.6 | Task Score | 33.7 |
| ToolBench - VirtualHome | 10.7 | Task Score | 25.6 |
| ToolBench - Trip Booking | 5 | Task Score | 24.4 |
| ToolBench Leaderboard | 15.94 | Average Task Score | 23.3 |
| ToolBench - The Cat API | 48 | Task Score | 22.1 |
| ToolBench - Open Weather | 37 | Task Score | 18.6 |
| Open LLM Leaderboard - IFEval | 23.71 | Score | 17.3 |
| Open LLM Leaderboard v1 - GSM8K | 1.52 | Accuracy (%) (5-shot) | 17.3 |
| ToolBench - Home Search | 4 | Task Score | 16.3 |
Interactive version: theaggregate.ai/model?slug=pythia-1-4b · How It Works · Data refreshed daily, snapshot 2026-10-20.