RPEval - Multi-Preference (Explicit Memory): leaderboard
RPEval (Feng et al., 2026) multi-preference setting: an underspecified Chinese-language query comes with several remembered preferences stated as explicit profile text, either all irrelevant or with one to three relevant ones, and the model labels each as to be ignored, supporting or dominating. Item-level accuracy in percent over 803 preferences in 150 records, most of which should be ignored, with the prompt reminding the model to judge relevance; higher is better.
Metric: Item-level (micro) accuracy (%; multi-preference items of RPEval: given a query and several explicit preferences, either all irrelevant (Ignore-All) or one to three relevant ones among irrelevant ones (Leave-K-Out), the share of preferences whose applicability the model classifies correctly; Reminder prompt). Source: arxiv.org. Saturation forecast: Around 2030. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | DeepSeek V3 | 56 | #705 |
| 2 | GPT-4.1 | 49 | #492 |
| 3 | GPT-5 | 39 | #163 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=rpeval-multi-preference-explicit-memory · How It Works · Data refreshed daily, snapshot 2026-11-12.