QCalEval (In-Context) - Significance Analysis: leaderboard

Scientific significance analysis on QCalEval (2026) quantum computing calibration plots with in-context demonstrations from the same experiment family, three expert key points per item scored by two LLM judges. Score from 0 to 100; higher is better.

Metric: Score (0-100) for the scientific significance analysis, three expert key points per item scored 0, 0.5 or 1 by GPT-5.4 and Gemini 3.1 Pro Preview and averaged, with one analysis demonstration per scenario type, on the QCalEval quantum-calibration plots with in-context demonstrations from the same experiment family (scenario types with a single sample left out), greedy decoding; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 17 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Claude Opus 4.684.7#113
2Gemini 3.1 Pro (Preview)81.3#99
3GPT-5.481#135
4Gemma 4 31B (IT)80.6#350
5Gemini 3.1 Flash Lite78.5#371
6Claude Sonnet 4.677.8#173
7Claude Haiku 4.566.1#583
8GPT-5.4 Mini58.8#406
9InternVL3-38B56.2#939
10InternVL3-78B50.5#759
11Qwen 3.5 27B41.8#375
12Qwen 3.5 397B A17B37.4#264
13Qwen 3.5 122B A10B36.1#346
14Kimi-VL-A3B34.9#2340
15Qwen 3.5 35B A3B33.4#492

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=qcaleval-in-context-significance-analysis · How It Works · Data refreshed daily, snapshot 2026-11-21.