M2-Verify-Med: leaderboard

M2-Verify (2026) is a multimodal scientific claim consistency benchmark: a model sees a figure, its caption and a claim and decides zero-shot whether the claim is supported or refuted. This board is macro-F1 in percent on the M2-Verify-Med test set, PubMed figures whose claims carry expert-curated medical perturbations; higher is better.

Metric: Macro-F1 (%) of binary claim consistency verification (supported or refuted) on the M2-Verify-Med test set (PubMed figures and captions, claims with expert-curated medical perturbations), zero-shot; higher is better. Source: arxiv.org. Saturation forecast: Around May 2027. 10 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1GPT-4o Mini73#1680
2Mistral Small 3.171.4#1734
3Qwen 2.5 VL 7B Instruct70.4#1919
4InternVL3-8B69.6#1795
5Pixtral-12B69.6#2611
6Llama 3.2 11B Vision69.3#4281
7GPT-5 Mini67.2#373
8Phi-4 Multimodal Instruct65.8#4155
9HuatuoGPT-Vision-7B60#2178
10LLaVA-Med51.8

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=m2-verify-med · How It Works · Data refreshed daily, snapshot 2026-11-22.