FeynmanBench - Propagator Inventory: leaderboard

FeynmanBench (2026) gives a multimodal model a machine-generated Feynman diagram (Standard Model QED, electroweak or QCD, tree level to two loops) with minimal labelling conventions and asks it to recover the full physical content of the diagram. Checkpoint 2: the complete list of internal lines with their particle types, checked by rule-based scripts; pass rate in percent, higher is better.

Metric: CP2 pass rate (%), the complete list of internal lines with particle types and multiplicities, checked by rule-based scripts, on FeynmanBench's Feynman-diagram tasks (a rendered diagram plus minimal conventions; Standard Model QED, electroweak and QCD diagrams from tree level to two loops), API models at temperature 0, mean of five runs; four models ran on subsets of 165 to 742 items, the others on about 2,000; higher is better. Source: arxiv.org. Saturation forecast: Not forecast. 17 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 3 Flash84.5#180
2Gemini 3.1 Pro (Preview)70.5#102
3Qwen 3.6 Plus49#245
4Qwen 3.5 Plus45#250
5Seed 2.0 Pro44.3#226
6Claude Sonnet 4.642.5#179
7GLM-5V Turbo30.1#365
8Claude Opus 4.716.6#86
9GPT-5.211.2#190
10GPT-5.110.7#248
11Claude Opus 4.59.6#165
12Claude Opus 4.69.4#114
13GPT-5.47.7#138
14Kimi K2.53.4#272
15Claude Sonnet 4.52#267

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=feynmanbench-propagator-inventory · How It Works · Data refreshed daily, snapshot 2026-11-22.