MechVQA - Structure Understanding: leaderboard
MechVQA (2026) tests multimodal LLMs on understanding mechanical engineering drawings, with 20,778 questions on 3,281 drawings scored as accuracy by three LLM judges against reference answers. This board covers the Structure Understanding subtask of the reasoning capability, as accuracy in percent on the test split; higher is better.
Metric: Accuracy (%) on the Structure Understanding subtask (Reasoning capability), MechVQA test split (drawing-level 8:1:1 split of 20,778 questions on 3,281 mechanical drawings), answers judged against the reference by three LLM judges (GPT-OSS-120B, DeepSeek-V3.2, Kimi-k2); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 15 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3 VL Plus | 81.25 | #512 |
| 2 | GPT-5 | 79.17 | #178 |
| 3 | GLM-4.6V | 77.08 | #710 |
| 4 | Claude Sonnet 4.5 | 70.83 | #261 |
| 5 | GPT-4o | 64.58 | #751 |
| 6 | Qwen 3 VL 32B Instruct | 62.5 | #606 |
| 7 | Gemini 3 Pro (Preview) | 52.08 | #119 |
| 8 | MiniCPM-V-4.5 | 45.83 | #1441 |
| 9 | Qwen 3 VL 30B A3B Instruct | 43.75 | #895 |
| 10 | Qwen 3 VL 4B Instruct | 39.58 | #1332 |
| 11 | InternVL3.5-8B | 37.5 | #1604 |
| 12 | Gemma 3 27B (IT) | 31.25 | #1324 |
| 13 | GPT-4o Mini | 29.17 | #1667 |
| 14 | MiMo-VL-7B-SFT | 27.08 | #1089 |
| 15 | Llama 3.2 11B Instruct | 6.25 | #5174 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=mechvqa-structure-understanding · How It Works · Data refreshed daily, snapshot 2026-11-21.