VeriOCRBench - Diagnosis Accuracy (Proactive): leaderboard
VeriOCRBench (Zhou et al., 2026) tests whether multimodal models notice that an OCR-grounded question cannot be answered, using 1,600 human-verified tasks with injected traps such as illegible or occluded text, missing targets, contradictory premises and missing variables. Diagnosis accuracy is the percentage of trap tasks on which the model both rejects the task and names the correct trap mechanism, judged by GPT-4o; higher is better. Proactive setting: the standard prompt, with no instruction to verify the task.
Metric: Diagnosis accuracy (%; share of VeriOCRBench's 1,600 trap-injected OCR reasoning tasks (8 trap types over visual, contextual, factual and logical premises, images from 8 OCR datasets, human-verified) on which the model both rejects the unexecutable task and names the ground-truth trap mechanism; judged by GPT-4o at temperature 0; model temperature 0 where supported; proactive paradigm: the standard image, textual premise and question with no verification instruction). Source: arxiv.org. Saturation forecast: Around 2029. 15 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4.5 | 46.88 | #244 |
| 2 | Qwen 3.5 35B A3B | 25.81 | #480 |
| 3 | Gemini 3.1 Pro (Preview) | 24.69 | #91 |
| 4 | Qwen 3.5 122B A10B | 23.56 | #329 |
| 5 | Qwen 3.5 397B A17B | 23.44 | #242 |
| 6 | Llama 4 Scout | 21.06 | #1787 |
| 7 | Llama 4 Maverick | 17.44 | #1105 |
| 8 | GPT-4o | 13.19 | #741 |
| 9 | InternVL3-38B | 12.69 | #905 |
| 10 | GPT-5 | 11.5 | #163 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=veriocrbench-diagnosis-accuracy-proactive · How It Works · Data refreshed daily, snapshot 2026-11-12.