SLMJury: leaderboard
Evaluates small language models (0.6B-14B) as judges: each model scores responses and its verdicts are compared against ground-truth labels, reporting the best judge accuracy per model across thinking-token-budget configurations.
Metric: Judge accuracy (%). Source: anishh15.github.io. Saturation forecast: Estimated already saturated. 16 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Phi-4 | 89.67 | #706 |
| 2 | Qwen 3 14B | 89.51 | #537 |
| 3 | Qwen 3 4B | 89.2 | #829 |
| 4 | Qwen 3 8B | 88.96 | #674 |
| 5 | Phi-4-reasoning-plus (Thinking) | 88.75 | |
| 6 | Phi-4 (Reasoning) | 88.24 | #706 (Phi-4) |
| 7 | Phi-4 Mini Instruct | 88.22 | #1245 |
| 8 | Llama 3.1 8B Instruct | 86.79 | #1035 |
| 9 | Qwen 2.5 7B Instruct | 86.62 | #862 |
| 10 | Qwen 3 1.7B | 85.96 | #1196 |
| 11 | Phi-4-mini (Reasoning) | 83.32 | |
| 12 | Llama 3.2 3B Instruct | 81.77 | #1352 |
| 13 | Qwen 2.5 1.5B Instruct | 79.91 | #1407 |
| 14 | Qwen 2.5 3B Instruct | 77.65 | #1162 |
| 15 | Qwen 3 0.6B | 76.37 | #1459 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=slmjury · How It Works · Data refreshed daily, snapshot 2026-10-20.