ReactBench (Hallucination) - Counterfactual Attribute (Standard): leaderboard

Accuracy in percent on ReactBench (2026) questions about images whose object attribute was edited against expectation, with standard prompting; an incorrect answer counts as a hallucination. Higher is better.

Metric: Accuracy (%) on Counterfactual Attribute (7,668 questions on 639 images with an edited attribute) with standard prompting, hallucination-inducing questions (multiple choice, short answer, yes/no) about images edited by Qwen-Image-Edit-2511; an incorrect answer counts as a hallucination; temperature 0.2, at most 1,024 tokens, LLM answer matching; higher is better. Source: arxiv.org. Saturation forecast: Around June 2027. 11 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Qwen 3.5 27B56.3#375
2Qwen 3 VL 32B Instruct55.4#606
3InternVL2.5-26B54.6#2156
4InternVL3-8B54.4#1782
5Qwen 3 VL 4B53.5#1151
6Qwen 2.5 VL 72B Instruct53.4#841
7InternVL2.5-8B50.7#2370
8Qwen 2.5 VL 7B Instruct49.2#1902
9Qwen 3.5 9B48.7#828
10ReactBench LLaVA-OV-7B (checkpoint unspecified)47.7
11ReactBench LLaVA-v1.6-7B (checkpoint unspecified)35.4

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=reactbench-hallucination-counterfactual-attribute-standard · How It Works · Data refreshed daily, snapshot 2026-11-21.