QuanBench+ - Cirq: leaderboard

Quantum code generation in Cirq on QuanBench+ (2026): 42 tasks on quantum algorithms, gate decomposition and state preparation, aligned across Qiskit, Cirq and PennyLane and checked by executable functional tests (probabilistic outputs accepted by KL divergence). Pass@1 with greedy decoding, in percent; higher is better.

Metric: Pass@1 (%): greedy decoding at temperature 0, one Cirq program per task, on the 42 framework-aligned QuanBench+ tasks (quantum algorithms, gate decomposition, state preparation), graded by executable functional tests with KL-divergence acceptance for probabilistic outputs; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 12 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 3 Pro54.8#146
2GPT-5.152.4#248
3DeepSeek R145.2#484
4Kimi K2 (Thinking)38.1
5GLM-4.738.1#383
6Gemini 2.5 Flash35.7#466
7Claude 3.7 Sonnet35.7#455
8GPT-4.133.3#506
9Llama 4 Maverick28.6#1113
10QuanBench+ DeepSeek-Chat (checkpoint unspecified)28.6
11MiniMax-M2.123.8#582
12Qwen 2.5 7B Instruct4.8#2732

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=quanbench-plus-cirq · How It Works · Data refreshed daily, snapshot 2026-11-22.