PolyChartQA (CoT) - MLM-Generated Questions: leaderboard

Question answering over multi-chart images on PolyChartQA (2026, Beyond Single Plots), machine-generated portion: 2,175 GPT-4.1-written, manually verified questions about multi-chart figures from computer science papers, answered with chain-of-thought prompting. Accuracy in percent rated by an LLM judge; higher is better.

Metric: L-Accuracy (%, times 100), chain-of-thought prompting, on the 2,175 GPT-4.1-generated, manually verified questions of PolyChartQA (534 multi-chart images with 2,297 sub-charts from computer science papers), answers rated correct by a Claude 3.7 Sonnet judge (L-Accuracy); higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 8 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 2.0 Flash81.06#731
2Claude 3.7 Sonnet74.71#455
3Pixtral-12B63.54#2611
4Gemma 3 4B (IT)25.29#4227
5ChartGemma23.36#8660
6Llama 3.2 11B Vision15.59#4281
7LLaVA-v1.5-7B11.59#7642
8MatCha4.41

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=polychartqa-cot-mlm-generated-questions · How It Works · Data refreshed daily, snapshot 2026-11-22.