ReactBench (Hallucination) - Counterfactual Attribute (CoT): leaderboard

Accuracy in percent on ReactBench (2026) questions about images whose object attribute was edited against expectation, with a step-by-step reasoning prompt; an incorrect answer counts as a hallucination. Higher is better.

Metric: Accuracy (%) on Counterfactual Attribute (7,668 questions on 639 images with an edited attribute) with a step-by-step reasoning prompt, hallucination-inducing questions (multiple choice, short answer, yes/no) about images edited by Qwen-Image-Edit-2511; an incorrect answer counts as a hallucination; temperature 0.2, at most 1,024 tokens, LLM answer matching; higher is better. Source: arxiv.org. Saturation forecast: Around May 2027. 13 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Qwen 3 VL 32B Instruct63.4#607
2InternVL3-8B60.9#1795
3Qwen 2.5 VL 72B Instruct60.7#849
4Qwen 3 VL 4B60.6#1163
5Qwen 3 VL 32B (Thinking)60.5#577
6InternVL2.5-26B59.7#2174
7Qwen 3.5 27B59.5#380
8Qwen 3.5 9B57.6#838
9Qwen 2.5 VL 7B Instruct55.2#1919
10ReactBench MiMo-VL-7B (checkpoint unspecified)50.8
11InternVL2.5-8B46.9#2393
12ReactBench LLaVA-OV-7B (checkpoint unspecified)44.8
13ReactBench LLaVA-v1.6-7B (checkpoint unspecified)37.5

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=reactbench-hallucination-counterfactual-attribute-cot · How It Works · Data refreshed daily, snapshot 2026-11-22.