DetailVerifyBench: leaderboard

Dense hallucination localization in long image captions on DetailVerifyBench (2026): the model receives an image and a caption of over 200 words that Gemini 3 Pro wrote for it, reproduces the caption and tags every hallucinated token; human annotators marked the real errors. 1,000 images from GUI, nature, chart, movie and poster domains; token-level F1 in percent, higher is better.

Metric: Token-level F1 (%, times 100) of hallucination localization on the real-hallucination version (Gemini 3 Pro captions corrected by human annotators; the hallucinated spans are the corrected ones): the model reproduces a long caption (over 200 words on average) of one of 1,000 images in five domains (GUI, nature, chart, movie, poster) and wraps every hallucinated token in tags, scored per token against human annotations; higher is better. Source: arxiv.org. Saturation forecast: Around 2030. 13 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Qwen 3.5 397B A17B13#264
2Gemini 3.1 Pro (Preview)12#99
3Seed 2.0 Pro11#222
4Qwen 3.5 35B A3B10#492
5Qwen 3.5 9B8#828
6Kimi K2.58#266
7GPT-5.47#135
8Claude Opus 4.67#113
9GPT-5.25#185
10Qwen 3 VL 8B (Thinking)3#910
11Step3 VL 10B3#1080
12MiMo-V2-Pro1#353
13GLM-4.6V-Flash1#1040

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=detailverifybench · How It Works · Data refreshed daily, snapshot 2026-11-21.