MemLens (128K) - Information Extraction: leaderboard

Accuracy in percent on the MemLens 246 questions that extract a fact or image detail from one earlier session, at a 128K-token interleaved image and text conversation history, with the evidence sessions hidden among distractor sessions (MemLens, 2026), judged by Qwen3-VL-235B. Higher is better.

Metric: Accuracy (%) on the information extraction (IE, 246 questions) at a 128K-token history, interleaved multimodal multi-session conversation history with the evidence sessions hidden among haystack sessions, judged correct or incorrect by a Qwen3-VL-235B-A22B-Instruct judge; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 25 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1GPT-5.460.16#138
2Qwen 3 VL 30B A3B Instruct56.5#903
3Gemini 3.1 Pro (Preview)55.79#102
4Qwen 3.5 27B54.07#380
5GLM-4.6V52.85#721
6Kimi K2.551.63#272
7Qwen 3 VL 235B A22B Instruct51.22#541
8Qwen 3.5 9B45.12#838
9Qwen 3.5 122B A10B43.09#350
10Cosmos-Reason2-8B36.99#758
11Qwen 3 VL 30B A3B (Thinking)33.74#727
12Qwen 3 VL 8B (Thinking)33.33#924
13Qwen 3 VL 8B Instruct32.52#1043
14Qwen 3.5 4B29.67#1270
15Qwen 3 VL 235B A22B (Thinking)29.27#462

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=memlens-128k-information-extraction · How It Works · Data refreshed daily, snapshot 2026-11-22.