HalluWorld - Terminal: leaderboard

Hallucination rate on HalluWorld-Terminal (2026): probes about recorded Terminal-Bench agent trajectories, with golden answers grounded in file-system diffs, covering perceptual, memory, causal, uncertainty and compound questions. Percent; lower is better.

Metric: Hallucination rate (%) over the 529 HalluWorld-Terminal probes about recorded Terminal-Bench agent trajectories (golden answers grounded in file-system diffs, context up to 60k characters), temperature 0 where supported, thinking models at medium effort with up to 16k thinking tokens, every model at most 256 answer tokens; lower is better. Source: arxiv.org. Saturation forecast: Around February 2027. 13 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-5.5Medium5.9#56
2O310.2#242
3O4 Mini14.2#372
4Claude Opus 4.6Medium15.7#114
5Claude Sonnet 4.6Medium17.8#179
6O3 Mini21#552
7GLM-5Non-reasoning26.3#276
8GPT-4o38.4#754
9DeepSeek V3.1Non-reasoning45.6#511
10GPT-5.4 MiniNon-reasoning50.3#409
11Qwen 3 30B A3BNon-reasoning51#1165
12GPT-4o Mini51.8#1680
13Kimi K2.6Non-reasoning56.5#197

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=halluworld-terminal · How It Works · Data refreshed daily, snapshot 2026-11-22.