MME-CoT - CoT Quality F1: leaderboard

Each model answers MME-CoT's visual questions step by step; on its 837 reasoning questions a GPT-4o judge matches the response against annotated key steps (recall) and checks each image observation and inference it states (precision), and the score is their F1. Unlike answer-accuracy boards such as MathVista, it grades the reasoning chain, not the final answer.

Metric: F1 (%). Source: arxiv.org. Saturation forecast: Around February 2027. 7 models tracked. Updated 17 Nov 2026.

Top models

#ModelScoreOverall rank
1QVQ-72B-Preview62#1173
2Virgo-72B60.8#1354
3InternVL2.5-78B-MPO52.7#1203
4InternVL2.5-8B-MPO43#2026
5InternVL2.5-8B41.1#2401
6MiniCPM-V-2.639.8#3523
7LLaVA-CoT34.9#3426

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=mme-cot-cot-quality-f1 · How It Works · Data refreshed daily, snapshot 2026-11-17.