Qwen2.5-Coder-1.5B: benchmark results
Alibaba's Qwen2.5-Coder-1.5B base model, part of the six-size Qwen2.5-Coder family (0.5B-32B) pretrained on 5.5 trillion tokens of code and text, with a 32,768-token context. Provider: Alibaba. Released 2024-09-19. Access: Open.
Aggregate Rating 1303 ± 33, rank #1473 of 1656 rated models, from 32 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| GSM8K | 65.8 | Accuracy (%) | 67.7 |
| MERA - ruHumanEval | 4.82 | pass@1 (%) | 36.8 |
| MERA - SimpleAr | 94.9 | EM (%) | 36.2 |
| MERA - BPS | 87.6 | Accuracy (%) | 33.3 |
| MERA - ruCodeEval | 1.95 | pass@1 (%) | 31.9 |
| ARC Challenge (AI2) | 45.2 | Accuracy (%) | 31.6 |
| MERA - RWSD | 52.31 | Accuracy (%) | 30.7 |
| MERA - ruModAr | 45.62 | EM (%) | 28.4 |
| MERA - ruMultiAr | 23.14 | EM (%) | 27.9 |
| C3-Bench (Controllable Code Completion) - Implementation Control | 19 | Instruction-Following Rate (%; 1,286 ICC tasks: passes the u | 27.5 |
| MMLU | 53.6 | Accuracy (%) | 23.9 |
| MERA - ruHateSpeech | 56.98 | Accuracy (%) | 22.6 |
Interactive version: theaggregate.ai/model?slug=qwen2-5-coder-1-5b · How It Works · Data refreshed daily, snapshot 2026-10-20.