DevBench - Unit Testing: leaderboard

Each model writes unit tests with the reference source code as input for 20 repositories (JavaScript has none); the score is the share of those tests that pass against the reference implementation, averaged uniformly over repositories. Unlike the acceptance-testing board it tests individual code units rather than the whole program.

Metric: Oracle Test (%). Source: raw.githubusercontent.com. Saturation forecast: Estimated already saturated. 9 models tracked. Updated 17 Nov 2026.

Top models

#ModelScoreOverall rank
1GPT-4 Preview (0125)36.5#1197
2GPT-4 Preview (1106)33.6#980
3DeepSeek-Coder-33B-Instruct32.8#2726
4GPT-3.5 Turbo (1106)28.7#3361
5deepseek-coder-6.7B-instruct23.5#7057
6CodeLlama-34B-Instruct21.1#7944
7deepseek-coder-1.3B-instruct5.6#9476
8CodeLlama-13B-Instruct5.1#8718
9CodeLlama-7B-Instruct3#9490

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=devbench-unit-testing · How It Works · Data refreshed daily, snapshot 2026-11-17.