QuanBench+ (Pass@5) - Cirq: leaderboard

Quantum code generation in Cirq on QuanBench+ (2026) with five samples per task at temperature 0.8: a task counts as solved when any of the five programs passes the executable functional tests of the 42 framework-aligned tasks. Pass@5 in percent; higher is better.

Metric: Pass@5 (%): any of five Cirq programs sampled at temperature 0.8 passes, on the 42 framework-aligned QuanBench+ tasks (quantum algorithms, gate decomposition, state preparation), graded by executable functional tests with KL-divergence acceptance for probabilistic outputs; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 12 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1GPT-5.164.3#248
2DeepSeek R161.9#484
3Gemini 3 Pro59.5#146
4GLM-4.759.5#383
5Claude 3.7 Sonnet57.1#455
6GPT-4.154.8#506
7Kimi K2 (Thinking)52.4
8MiniMax-M2.150#582
9QuanBench+ DeepSeek-Chat (checkpoint unspecified)45.2
10Llama 4 Maverick38.1#1113
11Gemini 2.5 Flash35.7#466
12Qwen 2.5 7B Instruct23.8#2732

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=quanbench-plus-pass-5-cirq · How It Works · Data refreshed daily, snapshot 2026-11-22.