PolyReal - Experiment Mechanism Reasoning: leaderboard

PolyReal (2026) evaluates multimodal models on 545 expert-written polymer science tasks covering five stages of real experimental work. Answers are scored against expert key points by an LLM judge and summarized as F1 from 0 to 1. This board is F1 on the module about reasoning about reaction diagrams and procedures; higher is better.

Metric: F1 score on the Experiment Mechanism Reasoning module of PolyReal (causal and procedural reasoning over reaction diagrams); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 15 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1O30.611#242
2GPT-50.608#185
3Claude Sonnet 4.5Thinking0.578#267
4Qwen 3 VL 235B A22B (Thinking)0.578#462
5Grok 40.577#321
6Qwen 3 VL 32B (Thinking)0.547#577
7Gemini 2.5 Pro0.535#280
8Gemini 2.5 Flash0.529#466
9DeepSeek R10.48#484
10Gemini 2.0 Flash (Thinking)0.466
11Intern-S10.403#627
12GPT-4o0.374#754
13Qwen 2.5 VL 72B Instruct0.312#849
14GPT-4o Mini0.258#1680
15DeepSeek-VL20.101#3056

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=polyreal-experiment-mechanism-reasoning · How It Works · Data refreshed daily, snapshot 2026-11-22.