AppellateGen - Reversal Prediction Accuracy: leaderboard
Whether the second-instance judgment a single model drafts in AppellateGen (Yang et al., 2026) reaches the correct ruling direction, affirm or reverse, on 1,000 Chinese appellate cases balanced between the two outcomes, as read from the generated judgment by a DeepSeek-V3.2 judge. Percent correct, with 50 as the chance level; higher is better.
Metric: Ruling-Direction Accuracy (%; affirm vs reverse, balanced 500/500). Source: arxiv.org. Saturation forecast: Around November 2027. 11 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 2.5 Flash (Non-reasoning) | 66.53 | #452 (Gemini 2.5 Flash) |
| 2 | Qwen 3 30B A3B 2507 (Thinking) | 64.81 | #759 |
| 3 | Qwen 3 235B A22B 2507 Instruct | 61.42 | #619 |
| 4 | Qwen 3 30B A3B 2507 Instruct | 60.43 | #1078 |
| 5 | QwQ-32B | 60.06 | #909 |
| 6 | Qwen 3 8B (Non-reasoning) | 57.92 | #1838 (Qwen 3 8B) |
| 7 | DeepSeek R1 0528 Qwen3 8B | 57.87 | #2101 |
| 8 | Gemini 2.5 Flash Lite (Non-reasoning) | 57.58 | #958 (Gemini 2.5 Flash Lite) |
| 9 | GPT-OSS-120B | 41.81 | #699 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=appellategen-reversal-prediction-accuracy · How It Works · Data refreshed daily, snapshot 2026-11-12.