QCalEval (In-Context) - Calibration Diagnosis: leaderboard

Choice of the family-specific calibration status code on QCalEval (2026) quantum computing calibration plots with in-context demonstrations from the same experiment family. Accuracy in percent; higher is better.

Metric: Accuracy (%) of the family-specific calibration status code, with one labeled demonstration per scenario type, on the QCalEval quantum-calibration plots with in-context demonstrations from the same experiment family (scenario types with a single sample left out), greedy decoding; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 17 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 3.1 Pro (Preview)89.8#99
2Claude Opus 4.689.4#113
3Gemma 4 31B (IT)86#350
4Gemini 3.1 Flash Lite82.2#371
5GPT-5.481.4#135
6Claude Sonnet 4.678#173
7Claude Haiku 4.573.1#583
8GPT-5.4 Mini66.9#406
9InternVL3-38B55.1#939
10Qwen 3.5 27B45.8#375
11InternVL3-78B44.3#759
12Qwen 3.5 397B A17B42.4#264
13Qwen 3.5 122B A10B35.2#346
14Qwen 3.5 9B33.9#828
15Qwen 3.5 35B A3B33.9#492

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=qcaleval-in-context-calibration-diagnosis · How It Works · Data refreshed daily, snapshot 2026-11-21.