MechVQA - Dimension and Annotation: leaderboard

MechVQA (2026) tests multimodal LLMs on understanding mechanical engineering drawings, with 20,778 questions on 3,281 drawings scored as accuracy by three LLM judges against reference answers. This board covers the Dimension and Annotation subtask of the recognition capability, as accuracy in percent on the test split; higher is better.

Metric: Accuracy (%) on the Dimension and Annotation subtask (Recognition capability), MechVQA test split (drawing-level 8:1:1 split of 20,778 questions on 3,281 mechanical drawings), answers judged against the reference by three LLM judges (GPT-OSS-120B, DeepSeek-V3.2, Kimi-k2); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 15 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 3 Pro (Preview)87.74#119
2Qwen 3 VL 32B Instruct86.68#606
3GLM-4.6V86.68#710
4Qwen 3 VL Plus86.68#512
5GPT-584.99#178
6Claude Sonnet 4.578.65#261
7Qwen 3 VL 30B A3B Instruct78.01#895
8Qwen 3 VL 4B Instruct76.96#1332
9MiMo-VL-7B-SFT76.74#1089
10MiniCPM-V-4.576.32#1441
11GPT-4o75.9#751
12GPT-4o Mini61.52#1667
13InternVL3.5-8B61.1#1604
14Gemma 3 27B (IT)55.39#1324
15Llama 3.2 11B Instruct38.05#5174

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=mechvqa-dimension-and-annotation · How It Works · Data refreshed daily, snapshot 2026-11-21.