UA-JudgeExam: leaderboard
Metric: Full-condition accuracy (%; question and four options shown, one letter answered, on a 400-item sample of the 8,128 items that remain after the Claude Haiku 4.5 option-only gate; denominators exclude unparseable answers (DeepSeek R1 352, Pixtral Large 365, Nova Micro 388, Llama 3.1 8B 202); greedy decoding where the provider accepts a temperature; chance is 25). Source: arxiv.org. Saturation forecast: Around December 2026. 13 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | GPT-5.6 Sol | 95.8 |
| 2 | Claude Sonnet 4.6 | 74 |
| 3 | DeepSeek R1 | 66.8 |
| 4 | Nova Pro | 58 |
| 5 | Claude Haiku 4.5 | 57.5 |
| 6 | Qwen 3 32B (Non-reasoning) | 55.2 |
| 7 | Llama 3.3 70B Instruct | 54 |
| 8 | Gemma 3 12B (IT) | 47.2 |
| 9 | Ministral-3-8B-Instruct-2512 | 46.3 |
| 10 | Nova 2.0 Lite (Non-reasoning) | 46 |
| 11 | Nova Micro | 43.6 |
Interactive version: theaggregate.ai/benchmark?slug=ua-judgeexam · How It Works · Data refreshed daily, snapshot 2026-09-29.