MechVQA - Identification and Counting: leaderboard

MechVQA (2026) tests multimodal LLMs on understanding mechanical engineering drawings, with 20,778 questions on 3,281 drawings scored as accuracy by three LLM judges against reference answers. This board covers the Identification and Counting subtask of the recognition capability, as accuracy in percent on the test split; higher is better.

Metric: Accuracy (%) on the Identification and Counting subtask (Recognition capability), MechVQA test split (drawing-level 8:1:1 split of 20,778 questions on 3,281 mechanical drawings), answers judged against the reference by three LLM judges (GPT-OSS-120B, DeepSeek-V3.2, Kimi-k2); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 15 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1GLM-4.6V88.37#710
2Qwen 3 VL 32B Instruct86.05#606
3Qwen 3 VL Plus81.4#512
4MiniCPM-V-4.579.07#1441
5MiMo-VL-7B-SFT79.07#1089
6Gemini 3 Pro (Preview)76.74#119
7Qwen 3 VL 30B A3B Instruct74.42#895
8GPT-569.77#178
9Claude Sonnet 4.567.44#261
10InternVL3.5-8B65.12#1604
11GPT-4o62.79#751
12Qwen 3 VL 4B Instruct62.79#1332
13Gemma 3 27B (IT)46.51#1324
14GPT-4o Mini32.56#1667
15Llama 3.2 11B Instruct13.95#5174

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=mechvqa-identification-and-counting · How It Works · Data refreshed daily, snapshot 2026-11-21.