QCalEval - Significance Analysis: leaderboard

Scientific significance analysis of each plot on QCalEval (2026), 243 quantum computing calibration examples from 22 experiment families of superconducting-qubit and neutral-atom experiments, answered zero-shot with the experiment background, three expert key points per item scored by two LLM judges. Score from 0 to 100; higher is better.

Metric: Score (0-100) for the scientific significance analysis, three expert key points per item scored 0, 0.5 or 1 by GPT-5.4 and Gemini 3.1 Pro Preview and averaged, on the 243 quantum-calibration examples of QCalEval (22 experiment families, superconducting qubits and neutral atoms), zero-shot with the family background text, greedy decoding; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 18 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Claude Opus 4.665.5#113
2Ising-Calibration-1-35B-A3B64.7
3GPT-5.463.7#135
4Claude Sonnet 4.663.4#173
5Gemini 3.1 Pro (Preview)61.1#99
6Gemma 4 31B (IT)59.8#350
7Gemini 3.1 Flash Lite59.4#371
8Qwen 3.5 397B A17B52#264
9Qwen 3.5 122B A10B49#346
10GPT-5.4 Mini48.3#406
11Qwen 3.5 27B48.3#375
12Qwen 3.5 35B A3B45.7#492
13Claude Haiku 4.540.8#583
14Qwen 3.5 9B39.5#828
15InternVL3-78B34.1#759

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=qcaleval-significance-analysis · How It Works · Data refreshed daily, snapshot 2026-11-21.