DetailVerifyBench - Synthetic Hallucinations: leaderboard

Localizing injected hallucinations in long image captions on DetailVerifyBench (2026): errors in number, color, category, shape, material, spatial relation, scene, camera, OCR and other details are injected into human-corrected captions and kept only when a text-only detector cannot spot them, so finding them requires the image. Token-level F1 in percent; higher is better.

Metric: Token-level F1 (%, times 100) of hallucination localization on the synthetic version (hallucinations injected into the corrected captions by Gemini 3 Flash and kept only when a text-only GPT-5.2 detector missed them, two adversarial rounds): the model reproduces a long caption (over 200 words on average) of one of 1,000 images in five domains (GUI, nature, chart, movie, poster) and wraps every hallucinated token in tags, scored per token against human annotations; higher is better. Source: arxiv.org. Saturation forecast: Around 2030. 14 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 3.1 Pro (Preview)66#99
2Gemini 3 Pro (Preview)64#119
3Qwen 3.5 397B A17B61#264
4Qwen 3.5 9B58#828
5Kimi K2.558#266
6Seed 2.0 Pro57#222
7Qwen 3.5 35B A3B52#492
8GPT-5.445#135
9GPT-5.243#185
10Qwen 3 VL 8B (Thinking)39#910
11Step3 VL 10B35#1080
12GLM-4.6V-Flash26#1040
13Claude Opus 4.69#113
14MiMo-V2-Pro3#353

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=detailverifybench-synthetic-hallucinations · How It Works · Data refreshed daily, snapshot 2026-11-21.