DeepSynth-Eval (Agentic Multi-Turn): leaderboard
DeepSynth-Eval (Zhang et al., Kuaishou Technology, 2026) asks for a survey paper on the topic of one of 96 published surveys, with structural constraints, and checks the report against 19,167 human-verified checklist requirements drawn from the reference survey (DeepSeek-V3.2 judge; incorrect statements subtract). This board uses a staged workflow: plan the sections, read selected full texts per section, write, then polish; the Overall score is a weighted percentage, higher is better.
Metric: Overall checklist score (%; plan, per-section selective full-text reading and writing, final polish; 96 survey-writing tasks, 19,167 atomic checklist requirements; group score min(1, net reward / saturation threshold) with -1 for an incorrect mention, weighted over groups; DeepSeek-V3.2 judge). Source: arxiv.org. Saturation forecast: Estimated already saturated. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3 235B A22B 2507 (Thinking) | 35.5 | #460 |
| 2 | GPT-5.2 | 33.3 | #172 |
| 3 | DeepSeek V3.2 | 30.4 | #403 |
| 4 | Qwen 3 30B A3B 2507 (Thinking) | 17.3 | #759 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=deepsynth-eval-agentic-multi-turn · How It Works · Data refreshed daily, snapshot 2026-11-12.