StratMem-Bench: leaderboard

StratMem-Bench (2026) tests strategic memory use in virtual-character conversation: from a pool of required, supportive and irrelevant memories, a reply must use every required memory, draw on at least one supportive memory when one is available, and leave the irrelevant ones out. This board is the percent of replies passing this strict check, over all instances weighted by scenario counts; higher is better.

Metric: Strict memory compliance SMC (%): share of virtual-character responses that use every must-use memory, at least one nice-to-use memory when available, and no irrelevant memories, over all instances weighted by scenario counts; higher is better. Source: arxiv.org. Saturation forecast: Around 2029. 9 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Claude Sonnet 4.551.45#261
2Gemini 3 Pro (Preview)50.68#119
3Llama 4 Maverick50.23#1110
4GPT-5.248.55#185
5Qwen 3 235B A22B 2507 Instruct47.18#616
6Qwen 3 Max46.42#415
7GPT-5 Chat46.27#449
8DeepSeek V3.245.66#419
9StratMem DeepSeek-reasoner (checkpoint unspecified)43.84

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=stratmem-bench · How It Works · Data refreshed daily, snapshot 2026-11-21.