PICQ-drama: leaderboard
Missing-persona identification for user simulation (Su et al., University of Tokyo, EACL 2026 Student Research Workshop): given a TV-drama dialogue ending in a choice question and the listener's basic persona, the model lists up to five persona dimensions that are absent but would decide the listener's answer, over 300 questions. Fidelity F1 in percent against the two annotators' merged personas, matched by persona category and an NLI entailment-or-contradiction test; higher is better. The prompt asks for the personas directly.
Metric: Fidelity F1 (%; up to five identified personas per question matched to the annotators' missing personas by category and NLI, baseline prompt). Source: arxiv.org. Saturation forecast: Estimated already saturated. 5 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3 32B | 46.8 | #960 |
| 2 | Llama 3.1 8B Instruct | 39.9 | #4277 |
| 3 | Qwen 3 8B | 39.3 | #1838 |
| 4 | GPT-4.1 | 37.4 | #492 |
| 5 | Llama 3.1 70B Instruct | 30.9 | #1541 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=picq-drama · How It Works · Data refreshed daily, snapshot 2026-11-12.