VitaBench 2.0 (Agentic Memory): leaderboard
VitaBench 2.0 (2026) personalized agent tasks with agentic memory: instead of reading the full interaction history, the model maintains its own memory of user preferences across temporally ordered task sequences and solves each task with tools in an executable environment. Mean success over four runs in percent, judged by a GPT-4.1 user simulator; higher is better.
Metric: Avg@4 (%, times 100): mean task success over four runs at temperature 0 of VitaBench 2.0 personalized agentic tasks (temporally ordered per-user task sequences across domains, tool-calling agent in an executable environment, preferences scattered over fragmented interaction histories, gpt-4.1-2025-04-14 as user simulator and rubric evaluator), with the interaction history kept in an agentic memory (MemAgent) that the model itself updates instead of the full context; higher is better. Source: arxiv.org. Saturation forecast: Around November 2027. 20 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Claude Opus 4.6 | Thinking | 45.4 | #113 |
| 2 | DeepSeek V4 Pro | 44.9 | #196 | |
| 3 | Seed 2.0 Pro | 42.8 | #222 | |
| 4 | GLM-5.1 | Non-reasoning | 42.3 | #198 |
| 5 | GPT-5 | 42.1 | #178 | |
| 6 | DeepSeek R1 0528 | 41.2 | #428 | |
| 7 | O3 | 40.1 | #237 | |
| 8 | Claude Sonnet 4.5 | Thinking | 39.7 | #261 |
| 9 | Kimi K2.6 | Non-reasoning | 39.7 | #192 |
| 10 | Seed-1.6 | 38.3 | #469 | |
| 11 | Gemini 2.5 Pro | 37.8 | #276 | |
| 12 | MiniMax-M2.7 | 35.1 | #539 | |
| 13 | GLM-4.6 | 35.1 | #498 | |
| 14 | GLM-4.5 | Non-reasoning | 33 | #542 |
| 15 | Qwen 3 Max | Thinking | 32.4 | #415 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=vitabench-2-0-agentic-memory · How It Works · Data refreshed daily, snapshot 2026-11-21.