Qwen 2.5 Coder 3B Instruct: benchmark results

Provider: Alibaba. Released 2024-11-12. Access: Open.

Aggregate Rating 1453 ± 24, rank #3737 of 10516 rated models, from 19 benchmark results.

Strongest benchmark results

BenchmarkScoreMetricPercentile
Structured Local-Deployment MCQ75.67Strict accuracy (%): share of the 1,085 items answered with 100
GSM8K80.7Accuracy (%)77.4
Delulu0.77pass@170
C3-Bench (Controllable Code Completion) - Scale Control14.2Instruction-Following Rate (%; 909 SCC tasks: AST node-type 65
JSONSchemaBench91.2Schema Compliance (percentage)57.1
Delulu - CodeBLEU47.2CodeBLEU (0-100): n-gram, syntax-tree and dataflow agreement55
C3-Bench (Controllable Code Completion) - Implementation Control29.7Instruction-Following Rate (%; 1,286 ICC tasks: passes the u52.5
Delulu - Edit Similarity69.3Edit similarity (0-100): character-level normalized Levensht50
Delulu - Exact Match44Exact match (%) of the completion with the gold completion, 50
LintEval - Pass@152Pass@1 (%; share of the 498 LintEval-Lite C++/Python/Java pr46.2
A.S.E (AICGSecEval) - Security29.52Code security score (%)33.3
A.S.E (AICGSecEval) - Quality38.28Code quality score (%)27.8

Interactive version: theaggregate.ai/model?slug=qwen-2-5-coder-3b-instruct · How It Works · Data refreshed daily, snapshot 2026-11-21.