FeynmanBench - Vertex Inventory: leaderboard

FeynmanBench (2026) gives a multimodal model a machine-generated Feynman diagram (Standard Model QED, electroweak or QCD, tree level to two loops) with minimal labelling conventions and asks it to recover the full physical content of the diagram. Checkpoint 1: the count and type of every interaction vertex, checked by rule-based scripts; pass rate in percent, higher is better.

Metric: CP1 pass rate (%), count and type of every interaction vertex with its linked fields, checked by rule-based scripts, on FeynmanBench's Feynman-diagram tasks (a rendered diagram plus minimal conventions; Standard Model QED, electroweak and QCD diagrams from tree level to two loops), API models at temperature 0, mean of five runs; four models ran on subsets of 165 to 742 items, the others on about 2,000; higher is better. Source: arxiv.org. Saturation forecast: Not forecast. 17 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 3 Flash94.7#175
2Gemini 3.1 Pro (Preview)85.7#99
3Seed 2.0 Pro79.1#222
4Qwen 3.6 Plus72.6#238
5Qwen 3.5 Plus66.7#242
6Claude Sonnet 4.665.2#173
7GLM-5V Turbo57.4#360
8GPT-5.146.7#240
9GPT-5.446.3#135
10GPT-5.239.9#185
11Claude Opus 4.538.6#160
12Claude Opus 4.632.6#113
13Claude Opus 4.731.4#85
14Qwen 3 VL Plus12.3#512
15Kimi K2.511.2#266

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=feynmanbench-vertex-inventory · How It Works · Data refreshed daily, snapshot 2026-11-21.