SantaCoder: benchmark results
Provider: Other. Access: Open.
Aggregate Rating 1131 ± 16, rank #10298 of 10916 rated models, from 70 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| MMLU-by-task - TruthfulQA MC2 | 51.24 | Accuracy (%) | 81.9 |
| ClassEval | 8.6 | Class-Level Pass@1 (%) | 60 |
| MMLU-by-task - Moral Scenarios | 26.37 | Accuracy (%) | 53.7 |
| Open LLM Leaderboard v1 - TruthfulQA MC2 | 51.24 | MC2 (%) (0-shot) | 51.2 |
| MMLU-by-task - High School Chemistry | 32.51 | Accuracy (%) | 47.4 |
| MMLU-by-task - High School Mathematics | 26.3 | Accuracy (%) | 40 |
| MMLU-by-task - TruthfulQA MC1 | 26.07 | Accuracy (%) | 39.7 |
| MMLU-by-task - Formal Logic | 27.78 | Accuracy (%) | 38.2 |
| MMLU-by-task - Human Aging | 37.67 | Accuracy (%) | 33.2 |
| MMLU-by-task - Security Studies | 35.1 | Accuracy (%) | 32.4 |
| MMLU-by-task - Conceptual Physics | 32.34 | Accuracy (%) | 31.4 |
| MMLU-by-task - Astronomy | 30.26 | Accuracy (%) | 31.2 |
Interactive version: theaggregate.ai/model?slug=santacoder · How It Works · Data refreshed daily, snapshot 2026-11-08.