starcoder2-15B: benchmark results
Provider: BigCode. Released 2024-02-28. Access: Open.
Unified ELO 1391 ± 1, rank #1174 of 1392 rated models, from 30 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| EffiBench - NET | 2.59 | Normalized Execution Time | 100 |
| EffiBench - NMU | 1.71 | Normalized Memory Usage | 100 |
| ShaderMatch | 18.78 | Clone Match Rate (%) | 100 |
| CRUXEval | 47.1 | Output Prediction pass@1 (%) | 66.7 |
| DomainCodeBench | 88.96 | Composite Score (%) | 66.7 |
| GSM8K | 57.7 | Accuracy (%) | 61.3 |
| Big Code Memorization - HumanEval-ET pass@1 | 40.24 | HumanEval-ET pass@1 (%) | 52.9 |
| EvoEval Tool Use | 48 | Pass@1 (%) | 51 |
| BigCode Models Leaderboard | 44.1 | HumanEval Python Pass@1 (%) | 49.2 |
| Big Code Memorization - HumanEval pass@1 | 46.95 | HumanEval pass@1 (%) | 47.1 |
| Big Code Memorization - HumanEval pass@50 | 34.06 | HumanEval pass@50 (%) | 47.1 |
| MMLU | 64.1 | Accuracy (%) | 41.5 |
Interactive version: theaggregate.ai/model?slug=starcoder2-15b · How It Works · Data refreshed daily, snapshot 2026-09-05.