QCalEval - Technical Description: leaderboard

Structured description of plot type, axes and salient features on QCalEval (2026), 243 quantum computing calibration examples from 22 experiment families of superconducting-qubit and neutral-atom experiments, answered zero-shot from the plot images, scored half by exact fields and half by key points judged by two LLM judges. Score from 0 to 100; higher is better.

Metric: Score (0-100) for a structured JSON description of plot type, axes and salient features (half exact-match fields, half key-point coverage judged by GPT-5.4 and Gemini 3.1 Pro Preview), on the 243 quantum-calibration examples of QCalEval (22 experiment families, superconducting qubits and neutral atoms), zero-shot from the plot images, greedy decoding; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 18 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1GPT-5.490.9#138
2Claude Opus 4.690.8#114
3GPT-5.4 Mini90.3#409
4Claude Sonnet 4.689.7#179
5Gemini 3.1 Flash Lite89.2#377
6Gemini 3.1 Pro (Preview)88.5#102
7Qwen 3.5 397B A17B88.1#270
8Ising-Calibration-1-35B-A3B87.8
9Qwen 3.5 27B87#380
10Qwen 3.5 35B A3B86.8#494
11Qwen 3.5 122B A10B86.6#350
12Gemma 4 31B (IT)85.6#354
13Claude Haiku 4.583.4#587
14Qwen 3.5 9B81.5#838
15InternVL3-38B79.2#948

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=qcaleval-technical-description · How It Works · Data refreshed daily, snapshot 2026-11-22.