deepseek-coder-33B-base: benchmark results
Provider: DeepSeek. Released 2023-10-28. Access: Open.
Aggregate Rating 1324 ± 46, rank #8271 of 10916 rated models, from 14 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| SAFIM | 69.01 | Average | 100 |
| CanItEdit (Descriptive) | 47.71 | pass@1 (%; share of the 105 hand-written Python editing prob | 79.2 |
| CanItEdit (Lazy) | 34.71 | pass@1 (%; share of the 105 hand-written Python editing prob | 75 |
| BigCode Models Leaderboard | 52.5 | HumanEval Python Pass@1 (%) | 66.1 |
| C3-Bench (Controllable Code Completion) - Implementation Control | 32 | Instruction-Following Rate (%; 1,286 ICC tasks: passes the u | 55 |
| C3-Bench (Controllable Code Completion) - Implementation Control - Pass@1 | 48.1 | Pass@1 (%; unit tests on the 1,286 instructed ICC tasks) | 50 |
| GSM8K | 35.4 | Accuracy (%) | 40.3 |
| EvalPlus (HumanEval+ & MBPP+) | 44.5 | Pass@1 avg (%) | 37.1 |
| C3-Bench (Controllable Code Completion) - Scale Control | 5.2 | Instruction-Following Rate (%; 909 SCC tasks: AST node-type | 36.3 |
| EffiBench - NET | 3.14 | Normalized Execution Time | 26.8 |
| ARC Challenge (AI2) | 42.2 | Accuracy (%) | 26.3 |
| WinoGrande | 62 | Accuracy (%) | 16.2 |
Interactive version: theaggregate.ai/model?slug=deepseek-coder-33b-base · How It Works · Data refreshed daily, snapshot 2026-11-08.