PolyChartQA (CoT) - MLM-Generated Questions: leaderboard
Question answering over multi-chart images on PolyChartQA (2026, Beyond Single Plots), machine-generated portion: 2,175 GPT-4.1-written, manually verified questions about multi-chart figures from computer science papers, answered with chain-of-thought prompting. Accuracy in percent rated by an LLM judge; higher is better.
Metric: L-Accuracy (%, times 100), chain-of-thought prompting, on the 2,175 GPT-4.1-generated, manually verified questions of PolyChartQA (534 multi-chart images with 2,297 sub-charts from computer science papers), answers rated correct by a Claude 3.7 Sonnet judge (L-Accuracy); higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 8 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 2.0 Flash | 81.06 | #731 |
| 2 | Claude 3.7 Sonnet | 74.71 | #455 |
| 3 | Pixtral-12B | 63.54 | #2611 |
| 4 | Gemma 3 4B (IT) | 25.29 | #4227 |
| 5 | ChartGemma | 23.36 | #8660 |
| 6 | Llama 3.2 11B Vision | 15.59 | #4281 |
| 7 | LLaVA-v1.5-7B | 11.59 | #7642 |
| 8 | MatCha | 4.41 |
No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=polychartqa-cot-mlm-generated-questions · How It Works · Data refreshed daily, snapshot 2026-11-22.