StratMem-Bench: leaderboard
StratMem-Bench (2026) tests strategic memory use in virtual-character conversation: from a pool of required, supportive and irrelevant memories, a reply must use every required memory, draw on at least one supportive memory when one is available, and leave the irrelevant ones out. This board is the percent of replies passing this strict check, over all instances weighted by scenario counts; higher is better.
Metric: Strict memory compliance SMC (%): share of virtual-character responses that use every must-use memory, at least one nice-to-use memory when available, and no irrelevant memories, over all instances weighted by scenario counts; higher is better. Source: arxiv.org. Saturation forecast: Around 2029. 9 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4.5 | 51.45 | #261 |
| 2 | Gemini 3 Pro (Preview) | 50.68 | #119 |
| 3 | Llama 4 Maverick | 50.23 | #1110 |
| 4 | GPT-5.2 | 48.55 | #185 |
| 5 | Qwen 3 235B A22B 2507 Instruct | 47.18 | #616 |
| 6 | Qwen 3 Max | 46.42 | #415 |
| 7 | GPT-5 Chat | 46.27 | #449 |
| 8 | DeepSeek V3.2 | 45.66 | #419 |
| 9 | StratMem DeepSeek-reasoner (checkpoint unspecified) | 43.84 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=stratmem-bench · How It Works · Data refreshed daily, snapshot 2026-11-21.