HalluWorld - Grid Memory: leaderboard

Memory hallucination in HalluWorld-Grid (2026), MiniGrid worlds whose ground truth comes from the environment state: the rate of hallucinated answers to questions about states seen in earlier observations. Percent; lower is better.

Metric: Hallucination rate (%) on the memory probes (tracking states across earlier observations, 6 levels) of HalluWorld-Grid (MiniGrid worlds whose ground truth comes from the environment state), micro-averaged over levels and serializers, temperature 0 where supported, thinking models at medium effort with up to 16k thinking tokens, every model at most 256 answer tokens; lower is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 13 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Claude Opus 4.63.8#114
2O4 Mini4.5#372
3O3 Mini4.9#552
4Claude Sonnet 4.65#179
5O35.5#242
6GPT-5.5Medium5.9#56
7Kimi K2.6Non-reasoning6.2#197
8DeepSeek V3 (0324)14.3#730
9GLM-5Non-reasoning15.8#276
10GPT-4o17.2#754
11GPT-5.4 MiniNon-reasoning26.6#409
12Qwen 3 30B A3BNon-reasoning27.7#1165
13GPT-4o Mini39#1680

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=halluworld-grid-memory · How It Works · Data refreshed daily, snapshot 2026-11-22.