VitaBench 2.0 (Agentic Memory): leaderboard

VitaBench 2.0 (2026) personalized agent tasks with agentic memory: instead of reading the full interaction history, the model maintains its own memory of user preferences across temporally ordered task sequences and solves each task with tools in an executable environment. Mean success over four runs in percent, judged by a GPT-4.1 user simulator; higher is better.

Metric: Avg@4 (%, times 100): mean task success over four runs at temperature 0 of VitaBench 2.0 personalized agentic tasks (temporally ordered per-user task sequences across domains, tool-calling agent in an executable environment, preferences scattered over fragmented interaction histories, gpt-4.1-2025-04-14 as user simulator and rubric evaluator), with the interaction history kept in an agentic memory (MemAgent) that the model itself updates instead of the full context; higher is better. Source: arxiv.org. Saturation forecast: Around November 2027. 20 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Claude Opus 4.6Thinking45.4#113
2DeepSeek V4 Pro44.9#196
3Seed 2.0 Pro42.8#222
4GLM-5.1Non-reasoning42.3#198
5GPT-542.1#178
6DeepSeek R1 052841.2#428
7O340.1#237
8Claude Sonnet 4.5Thinking39.7#261
9Kimi K2.6Non-reasoning39.7#192
10Seed-1.638.3#469
11Gemini 2.5 Pro37.8#276
12MiniMax-M2.735.1#539
13GLM-4.635.1#498
14GLM-4.5Non-reasoning33#542
15Qwen 3 MaxThinking32.4#415

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=vitabench-2-0-agentic-memory · How It Works · Data refreshed daily, snapshot 2026-11-21.