MechVQA - Anomaly Detection: leaderboard
MechVQA (2026) tests multimodal LLMs on understanding mechanical engineering drawings, with 20,778 questions on 3,281 drawings scored as accuracy by three LLM judges against reference answers. This board covers the Anomaly Detection subtask of the judging capability, as accuracy in percent on the test split; higher is better.
Metric: Accuracy (%) on the Anomaly Detection subtask (Judging capability), MechVQA test split (drawing-level 8:1:1 split of 20,778 questions on 3,281 mechanical drawings), answers judged against the reference by three LLM judges (GPT-OSS-120B, DeepSeek-V3.2, Kimi-k2); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 15 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3 Pro (Preview) | 78.37 | #119 |
| 2 | Qwen 3 VL 32B Instruct | 75.92 | #606 |
| 3 | GLM-4.6V | 74.29 | #710 |
| 4 | GPT-5 | 71.02 | #178 |
| 5 | Qwen 3 VL Plus | 67.76 | #512 |
| 6 | Claude Sonnet 4.5 | 64.9 | #261 |
| 7 | Qwen 3 VL 30B A3B Instruct | 64.08 | #895 |
| 8 | Qwen 3 VL 4B Instruct | 62.86 | #1332 |
| 9 | MiniCPM-V-4.5 | 60 | #1441 |
| 10 | MiMo-VL-7B-SFT | 59.59 | #1089 |
| 11 | InternVL3.5-8B | 53.47 | #1604 |
| 12 | GPT-4o | 53.06 | #751 |
| 13 | GPT-4o Mini | 35.1 | #1667 |
| 14 | Gemma 3 27B (IT) | 24.9 | #1324 |
| 15 | Llama 3.2 11B Instruct | 17.55 | #5174 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=mechvqa-anomaly-detection · How It Works · Data refreshed daily, snapshot 2026-11-21.