MemLens (64K): leaderboard

MemLens tests the multimodal long-term memory of vision-language models with 789 questions whose answers depend on images and facts spread over many sessions. Overall accuracy in percent at a 64K-token interleaved image and text conversation history, with the evidence sessions hidden among distractor sessions (MemLens, 2026), judged by Qwen3-VL-235B; higher is better.

Metric: Overall accuracy (%) on all 789 questions (information extraction, multi-session, temporal and knowledge-update reasoning and answer refusal) at a 64K-token history, interleaved multimodal multi-session conversation history with the evidence sessions hidden among haystack sessions, judged correct or incorrect by a Qwen3-VL-235B-A22B-Instruct judge; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 27 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Qwen 3.5 122B A10B55.89#346
2Kimi K2.553.99#266
3Gemini 3.1 Pro (Preview)52.99#99
4GPT-5.452.34#135
5Qwen 3 VL 235B A22B Instruct50.82#532
6Qwen 3 VL 30B A3B Instruct49.81#895
7GLM-4.6V49.05#710
8Qwen 3.5 27B46.13#375
9Qwen 3.5 9B44.36#828
10Qwen 3 VL 8B Instruct42.84#1029
11Nemotron Nano 12B v2 VL40.94#2368
12Qwen 3 VL 8B (Thinking)39.67#910
13Qwen 3 VL 235B A22B (Thinking)39.67#459
14Cosmos-Reason2-8B39.16#755
15Qwen 3.5 4B37.14#1241

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=memlens-64k · How It Works · Data refreshed daily, snapshot 2026-11-21.