RefMem-Bench - Direct-Answer: leaderboard
Reflective memory in long multi-session dialogues on RefMem-Bench (Lin et al., 2026, Connecting the Dots): questions ask for latent interpretations such as beliefs, habits, boundaries and plans that must be pieced together from cues scattered across the history, with images given as indexed captions. Short free-form answers judged equivalent to the references by a GPT-4o judge; accuracy in percent averaged over eight reflective-memory dimensions on the test split, higher is better.
Metric: Answer accuracy (%; short free-form answers judged equivalent to a reference by a GPT-4o judge; mean of the eight reflective-memory dimensions over the 20,279-question test split; dialogue history with image captions in context, temperature 0). Source: arxiv.org. Saturation forecast: Around February 2027. 5 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | DeepSeek R1 Distill Qwen 14B | 19.1 | #2594 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=refmem-bench-direct-answer · How It Works · Data refreshed daily, snapshot 2026-11-12.