RPEval - Single Preference (Explicit Memory): leaderboard
RPEval (Feng et al., 2026) tests rational use of personal memory on Chinese-language queries: given an underspecified query and one remembered user preference stated as explicit profile text, the model decides whether the preference should be ignored, support the answer or dominate it. Accuracy in percent on 150 human-verified items balanced across the three labels, with the prompt reminding the model to judge relevance; higher is better.
Metric: Intent-matching accuracy (%; single-preference items of RPEval's human-verified test set: given one explicit user preference and an underspecified query, the model classifies whether the preference should be ignored, support the answer or dominate it; Reminder prompt asking the model to judge the memory's applicability). Source: arxiv.org. Saturation forecast: Around March 2028. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | DeepSeek V3 | 66 | #705 |
| 2 | GPT-4.1 | 53 | #492 |
| 3 | GPT-5 | 51 | #163 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=rpeval-single-preference-explicit-memory · How It Works · Data refreshed daily, snapshot 2026-11-12.