LongJudgeBench (Rubric) - RealDeepResearch: leaderboard
LongJudgeBench (2026) measures how often an LLM judge agrees with human judgments on long-form outputs averaging over 9,000 tokens: deep research reports, scientific surveys, creative writing, long reasoning chains and systematic reviews. This board is judge accuracy in percent on RealDeepResearch, 640 expert-annotated Chinese and English deep-research reports in their original DOCX form, the judge working given task-specific rubrics; higher is better.
Metric: Judge accuracy (%) on RealDeepResearch (640 expert-annotated Chinese and English deep-research reports in DOCX form, pointwise): agreement of the LLM judge's pairwise preferences (from pointwise scores, pairwise choices or listwise ranks) or correctness labels with the human reference judgments, averaged within query groups, temperature 0, judging with task-specific rubrics; higher is better. Source: arxiv.org. Saturation forecast: Around March 2027. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | DeepSeek V4 Flash | 57.1 | #258 |
| 2 | GLM-5.1 | 56.06 | #179 |
| 3 | Kimi K2.6 | 51.75 | #178 |
| 4 | Qwen 3 Max | 50.19 | #403 |
| 5 | Qwen 3 32B (Thinking) | 50.17 | #960 (Qwen 3 32B) |
| 6 | GPT-5.2 | 44.29 | #176 |
| 7 | Qwen 3 32B (Non-reasoning) | 38.81 | #960 (Qwen 3 32B) |
| 8 | GPT-4o Mini | 34.58 | #1623 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=longjudgebench-rubric-realdeepresearch · How It Works · Data refreshed daily, snapshot 2026-11-13.