MemLens (64K) - Information Extraction: leaderboard

Accuracy in percent on the MemLens 246 questions that extract a fact or image detail from one earlier session, at a 64K-token interleaved image and text conversation history, with the evidence sessions hidden among distractor sessions (MemLens, 2026), judged by Qwen3-VL-235B. Higher is better.

Metric: Accuracy (%) on the information extraction (IE, 246 questions) at a 64K-token history, interleaved multimodal multi-session conversation history with the evidence sessions hidden among haystack sessions, judged correct or incorrect by a Qwen3-VL-235B-A22B-Instruct judge; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 27 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Qwen 3.5 122B A10B67.07#350
2GPT-5.463.01#138
3Qwen 3.5 27B62.2#380
4GLM-4.6V61.79#721
5Qwen 3.5 9B60.57#838
6Gemini 3.1 Pro (Preview)58.94#102
7Qwen 3 VL 30B A3B Instruct56.91#903
8Qwen 3 VL 235B A22B Instruct56.1#541
9Nemotron Nano 12B v2 VL56.1#2385
10Kimi K2.552.44#272
11Qwen 3.5 4B49.19#1270
12Cosmos-Reason2-8B46.34#758
13GLM-4.5V45.93#756
14Qwen 3 VL 8B Instruct44.72#1043
15Qwen 3 VL 4B Instruct42.28#1348

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=memlens-64k-information-extraction · How It Works · Data refreshed daily, snapshot 2026-11-22.