MemLens (32K) - Knowledge Update: leaderboard

Accuracy in percent on the MemLens 116 questions whose answer changed in a later session, at a 32K-token interleaved image and text conversation history, with the evidence sessions hidden among distractor sessions (MemLens, 2026), judged by Qwen3-VL-235B. Higher is better.

Metric: Accuracy (%) on the knowledge update (KU, 116 questions) at a 32K-token history, interleaved multimodal multi-session conversation history with the evidence sessions hidden among haystack sessions, judged correct or incorrect by a Qwen3-VL-235B-A22B-Instruct judge; higher is better. Source: arxiv.org. Saturation forecast: Around September 2028. 27 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Kimi K2.550.86#266
2Gemini 3.1 Pro (Preview)49.14#99
3Qwen 3.5 122B A10B49.14#346
4GPT-5.447.41#135
5GLM-4.6V43.97#710
6Qwen 3.5 9B43.1#828
7Qwen 3.5 27B42.24#375
8Qwen 3 VL 235B A22B Instruct40.52#532
9Nemotron Nano 12B v2 VL39.66#2368
10Qwen 3 VL 30B A3B Instruct37.93#895
11Qwen 3 VL 235B A22B (Thinking)36.21#459
12Qwen 3.5 4B34.48#1241
13Qwen 3 VL 8B Instruct33.62#1029
14Qwen 3.5 2B33.62#5307
15Qwen 3 VL 4B (Thinking)33.62#1334

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=memlens-32k-knowledge-update · How It Works · Data refreshed daily, snapshot 2026-11-21.