HalluWorld - Grid Causal: leaderboard

Causal hallucination in HalluWorld-Grid (2026), MiniGrid worlds whose ground truth comes from the environment state: the rate of hallucinated answers to questions that need forward simulation of fire, flood and pressure-plate mechanics. Percent; lower is better.

Metric: Hallucination rate (%) on the causal probes (forward simulation of fire, flood and pressure-plate mechanics, 9 levels) of HalluWorld-Grid (MiniGrid worlds whose ground truth comes from the environment state), micro-averaged over levels and serializers, temperature 0 where supported, thinking models at medium effort with up to 16k thinking tokens, every model at most 256 answer tokens; lower is better. Source: arxiv.org. Saturation forecast: Around January 2028. 13 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Claude Sonnet 4.619.7#173
2Claude Opus 4.620.1#113
3GPT-5.4 MiniNon-reasoning22.2#406
4O3 Mini23.8#546
5GPT-5.5Medium25.3#54
6O326#237
7GLM-5Non-reasoning26.4#269
8GPT-4o29#751
9O4 Mini30.6#368
10GPT-4o Mini30.9#1667
11Qwen 3 30B A3BNon-reasoning31.6#1156
12DeepSeek V3 (0324)32.2#715
13Kimi K2.6Non-reasoning47.4#192

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=halluworld-grid-causal · How It Works · Data refreshed daily, snapshot 2026-11-21.