UA-JudgeExam: leaderboard
UA-JudgeExam (2026) is built from the question bank the Ukrainian High Qualification Commission of Judges publishes for appellate-judge candidates: four-option legal items with official keys, filtered to remove items a model can answer from the options alone. Score is multiple-choice accuracy with the question shown, on a 400-item sample of the filtered set, in percent; chance is 25 and higher is better.
Metric: Full-condition accuracy (%; question and four options shown, one letter answered, on a 400-item sample of the 8,128 items that remain after the Claude Haiku 4.5 option-only gate; denominators exclude unparseable answers (DeepSeek R1 352, Pixtral Large 365, Nova Micro 388, Llama 3.1 8B 202); greedy decoding where the provider accepts a temperature; chance is 25). Source: arxiv.org. Saturation forecast: Estimated already saturated. 13 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.6 Sol | 95.8 | #30 |
| 2 | Claude Sonnet 4.6 | 74 | #161 |
| 3 | DeepSeek R1 | 66.8 | #473 |
| 4 | Nova Pro | 58 | #1555 |
| 5 | Claude Haiku 4.5 | 57.5 | #568 |
| 6 | Qwen 3 32B (Non-reasoning) | 55.2 | #960 (Qwen 3 32B) |
| 7 | Llama 3.3 70B Instruct | 54 | #1357 |
| 8 | Gemma 3 12B (IT) | 47.2 | #1837 |
| 9 | Ministral-3-8B-Instruct-2512 | 46.3 | #1858 |
| 10 | Nova 2.0 Lite (Non-reasoning) | 46 | #1126 (Nova 2.0 Lite) |
| 11 | Nova Micro | 43.6 | #2717 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=ua-judgeexam · How It Works · Data refreshed daily, snapshot 2026-11-13.