QuanBench+ (Feedback Repair) - Cirq: leaderboard

Quantum code generation in Cirq with feedback repair on QuanBench+ (2026): after a runtime error or a wrong answer on one of the 42 framework-aligned tasks, the model receives the exception trace or its failing function and may submit a corrected program, up to five times. Pass@1 after repair, in percent; higher is better.

Metric: Pass@1 after feedback repair (%): after a runtime error the model receives the exception trace, after a wrong answer its failing function, and returns a corrected Cirq program, up to five repair attempts per task, on the 42 framework-aligned QuanBench+ tasks (quantum algorithms, gate decomposition, state preparation), graded by executable functional tests with KL-divergence acceptance for probabilistic outputs; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 12 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 3 Pro76.2#145
2GPT-5.173.8#240
3DeepSeek R161.9#482
4GLM-4.761.9#380
5Claude 3.7 Sonnet59.5#470
6GPT-4.157.1#505
7Kimi K2 (Thinking)57.1
8Gemini 2.5 Flash50#463
9MiniMax-M2.147.6#575
10Llama 4 Maverick42.9#1110
11QuanBench+ DeepSeek-Chat (checkpoint unspecified)40.5
12Qwen 2.5 7B Instruct7.1#2723

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=quanbench-plus-feedback-repair-cirq · How It Works · Data refreshed daily, snapshot 2026-11-21.