VeriOCRBench - Diagnosis Accuracy (Assisted): leaderboard
VeriOCRBench (Zhou et al., 2026) tests whether multimodal models notice that an OCR-grounded question cannot be answered, using 1,600 human-verified tasks with injected traps such as illegible or occluded text, missing targets, contradictory premises and missing variables. Diagnosis accuracy is the percentage of trap tasks on which the model both rejects the task and names the correct trap mechanism, judged by GPT-4o; higher is better. Assisted setting: the prompt tells the model to check that the task is completable before answering.
Metric: Diagnosis accuracy (%; share of VeriOCRBench's 1,600 trap-injected OCR reasoning tasks (8 trap types over visual, contextual, factual and logical premises, images from 8 OCR datasets, human-verified) on which the model both rejects the unexecutable task and names the ground-truth trap mechanism; judged by GPT-4o at temperature 0; model temperature 0 where supported; assisted paradigm: a meta-instruction asks the model to verify that the task is completable before answering). Source: arxiv.org. Saturation forecast: Around May 2027. 15 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3.5 397B A17B | 80.5 | #242 |
| 2 | GPT-5 | 76.94 | #163 |
| 3 | Claude Sonnet 4.5 | 76.12 | #244 |
| 4 | Qwen 3.5 35B A3B | 74.19 | #480 |
| 5 | Qwen 3.5 122B A10B | 73.81 | #329 |
| 6 | GPT-4o | 67.38 | #741 |
| 7 | InternVL3-38B | 67.31 | #905 |
| 8 | Llama 4 Maverick | 60.88 | #1105 |
| 9 | Llama 4 Scout | 54.62 | #1787 |
| 10 | Gemini 3.1 Pro (Preview) | 7.25 | #91 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=veriocrbench-diagnosis-accuracy-assisted · How It Works · Data refreshed daily, snapshot 2026-11-12.