starcoder — benchmark results
Provider: BigCode. Released 2023-04-24. Access: Open.
Unified ELO 1198 ± 20, rank #1693 of 1776 rated models, from 92 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| ToolBench - WebShop Long | 2.6 | Task Score | 97.7 |
| EffiBench - NET | 2.84 | Normalized Execution Time | 81.7 |
| ToolBench - Google Sheets | 48 | Task Score | 74.4 |
| ToolBench - Open Weather | 91 | Task Score | 74.4 |
| ToolBench - Home Search | 82 | Task Score | 68.6 |
| MMLU-by-task - Econometrics | 30.7 | Accuracy (%) | 68.2 |
| ToolBench - Tabletop | 21.9 | Task Score | 67.4 |
| MMLU-by-task - College Mathematics | 33 | Accuracy (%) | 61.7 |
| ToolBench - VirtualHome | 19.4 | Task Score | 60.5 |
| ToolBench Leaderboard | 49.75 | Average Task Score | 60.5 |
| EffiBench - NMU | 1.91 | Normalized Memory Usage | 58.5 |
| CodeScope - Code Translation | 3.85 | Score | 57.1 |
Interactive version: theaggregate.ai/model?slug=starcoder · How the rankings work · Data refreshed daily, snapshot 2026-07-22.