DO-Bench - Prior Robustness: leaderboard

Resistance to misleading context in object-existence questions on DO-Bench (2026): across four levels of textual prior pressure, the model must keep affirming a present but unusual object and keep denying an expected object that was removed from the image. PriorRobust in percent (any constant answer scores 50); higher is better.

Metric: PriorRobust (%): one minus the normalized area under the false-negative curve (present object, prior strength 0-3) and under the false-positive curve (absent object, prior strength 0-3), averaged; any constant answer scores 50, on DO-Bench, 124 scenes with ten controlled yes/no object-existence queries each (1,240 queries: a present-but-anomalous object under four prior strengths and two zoomed views, an absent-but-expected object removed by local inpainting under four prior strengths), expert-verified images, temperature 0 or the mean of three API runs; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 17 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 3 Flash (Preview)91.45#156
2GPT-5.284.64#190
3Claude Opus 4.681.45#114
4InternVL2.5-8B65.72#2393
5InternVL2.5-78B63.78#1036
6Qwen 2.5 VL 32B Instruct63.31#1065
7Qwen 2.5 VL 72B Instruct61.62#849
8InternVL2.5-38B58.87#1383
9Qwen 2.5 VL 7B Instruct56.45#1919
10Qwen 2.5 VL 3B Instruct53.36#2720
11InternVL2.5-4B49.8#2573
12InternVL2.5-2B36.7#6523
13LLaVA-v1.5-7B21.78#7642
14Qwen-VL-7B21.25
15InstructBLIP-Vicuna-13B14.04#8813

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=do-bench-prior-robustness · How It Works · Data refreshed daily, snapshot 2026-11-22.