MechVQA - Geometric Calculation: leaderboard

MechVQA (2026) tests multimodal LLMs on understanding mechanical engineering drawings, with 20,778 questions on 3,281 drawings scored as accuracy by three LLM judges against reference answers. This board covers the Geometric Calculation subtask of the reasoning capability, as accuracy in percent on the test split; higher is better.

Metric: Accuracy (%) on the Geometric Calculation subtask (Reasoning capability), MechVQA test split (drawing-level 8:1:1 split of 20,778 questions on 3,281 mechanical drawings), answers judged against the reference by three LLM judges (GPT-OSS-120B, DeepSeek-V3.2, Kimi-k2); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 15 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1GLM-4.6V82.93#710
2Qwen 3 VL Plus79.67#512
3Qwen 3 VL 32B Instruct76.83#606
4Claude Sonnet 4.575.2#261
5GPT-573.58#178
6Gemini 3 Pro (Preview)73.58#119
7Qwen 3 VL 30B A3B Instruct64.23#895
8GPT-4o63.41#751
9MiniCPM-V-4.559.76#1441
10Qwen 3 VL 4B Instruct58.54#1332
11MiMo-VL-7B-SFT54.47#1089
12Gemma 3 27B (IT)51.22#1324
13InternVL3.5-8B48.78#1604
14GPT-4o Mini47.97#1667
15Llama 3.2 11B Instruct25.61#5174

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=mechvqa-geometric-calculation · How It Works · Data refreshed daily, snapshot 2026-11-21.