MemLens (32K) - Information Extraction: leaderboard

Accuracy in percent on the MemLens 246 questions that extract a fact or image detail from one earlier session, at a 32K-token interleaved image and text conversation history, with the evidence sessions hidden among distractor sessions (MemLens, 2026), judged by Qwen3-VL-235B. Higher is better.

Metric: Accuracy (%) on the information extraction (IE, 246 questions) at a 32K-token history, interleaved multimodal multi-session conversation history with the evidence sessions hidden among haystack sessions, judged correct or incorrect by a Qwen3-VL-235B-A22B-Instruct judge; higher is better. Source: arxiv.org. Saturation forecast: Around July 2027. 26 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Qwen 3.5 122B A10B74.39#346
2Qwen 3.5 27B70.33#375
3GPT-5.469.51#135
4Qwen 3.5 9B67.48#828
5Qwen 3 VL 30B A3B Instruct65.45#895
6Qwen 3.5 2B64.23#5307
7Nemotron Nano 12B v2 VL63.82#2368
8GLM-4.5V61.79#754
9Qwen 3 VL 235B A22B Instruct60.98#532
10GLM-4.6V59.35#710
11Gemini 3.1 Pro (Preview)57.32#99
12Qwen 3.5 4B56.91#1241
13Cosmos-Reason2-8B55.69#755
14Qwen 3 VL 8B Instruct52.85#1029
15Qwen 3 VL 4B Instruct52.03#1332

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=memlens-32k-information-extraction · How It Works · Data refreshed daily, snapshot 2026-11-21.