MechVQA - Identification and Counting: leaderboard
MechVQA (2026) tests multimodal LLMs on understanding mechanical engineering drawings, with 20,778 questions on 3,281 drawings scored as accuracy by three LLM judges against reference answers. This board covers the Identification and Counting subtask of the recognition capability, as accuracy in percent on the test split; higher is better.
Metric: Accuracy (%) on the Identification and Counting subtask (Recognition capability), MechVQA test split (drawing-level 8:1:1 split of 20,778 questions on 3,281 mechanical drawings), answers judged against the reference by three LLM judges (GPT-OSS-120B, DeepSeek-V3.2, Kimi-k2); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 15 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GLM-4.6V | 88.37 | #710 |
| 2 | Qwen 3 VL 32B Instruct | 86.05 | #606 |
| 3 | Qwen 3 VL Plus | 81.4 | #512 |
| 4 | MiniCPM-V-4.5 | 79.07 | #1441 |
| 5 | MiMo-VL-7B-SFT | 79.07 | #1089 |
| 6 | Gemini 3 Pro (Preview) | 76.74 | #119 |
| 7 | Qwen 3 VL 30B A3B Instruct | 74.42 | #895 |
| 8 | GPT-5 | 69.77 | #178 |
| 9 | Claude Sonnet 4.5 | 67.44 | #261 |
| 10 | InternVL3.5-8B | 65.12 | #1604 |
| 11 | GPT-4o | 62.79 | #751 |
| 12 | Qwen 3 VL 4B Instruct | 62.79 | #1332 |
| 13 | Gemma 3 27B (IT) | 46.51 | #1324 |
| 14 | GPT-4o Mini | 32.56 | #1667 |
| 15 | Llama 3.2 11B Instruct | 13.95 | #5174 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=mechvqa-identification-and-counting · How It Works · Data refreshed daily, snapshot 2026-11-21.