RPEval - Multi-Preference (Implicit Memory): leaderboard
RPEval (Feng et al., 2026) multi-preference setting: an underspecified Chinese-language query comes with several remembered preferences embedded in earlier dialogue turns, either all irrelevant or with one to three relevant ones, and the model labels each as to be ignored, supporting or dominating. Item-level accuracy in percent over 803 preferences in 150 records, most of which should be ignored, with the prompt reminding the model to judge relevance; higher is better.
Metric: Item-level (micro) accuracy (%; multi-preference items of RPEval: given a query and several implicit (dialogue-embedded) preferences, either all irrelevant (Ignore-All) or one to three relevant ones among irrelevant ones (Leave-K-Out), the share of preferences whose applicability the model classifies correctly; Reminder prompt). Source: arxiv.org. Saturation forecast: Around 2028. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | DeepSeek V3 | 57 | #705 |
| 2 | GPT-5 | 39 | #163 |
| 3 | GPT-4.1 | 26 | #492 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=rpeval-multi-preference-implicit-memory · How It Works · Data refreshed daily, snapshot 2026-11-12.