QuanBench+ (Feedback Repair) - Qiskit: leaderboard

Quantum code generation in Qiskit with feedback repair on QuanBench+ (2026): after a runtime error or a wrong answer on one of the 42 framework-aligned tasks, the model receives the exception trace or its failing function and may submit a corrected program, up to five times. Pass@1 after repair, in percent; higher is better.

Metric: Pass@1 after feedback repair (%): after a runtime error the model receives the exception trace, after a wrong answer its failing function, and returns a corrected Qiskit program, up to five repair attempts per task, on the 42 framework-aligned QuanBench+ tasks (quantum algorithms, gate decomposition, state preparation), graded by executable functional tests with KL-divergence acceptance for probabilistic outputs; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 12 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1GPT-5.183.3#248
2Gemini 3 Pro73.8#146
3DeepSeek R171.4#484
4Kimi K2 (Thinking)69
5GLM-4.769#383
6Gemini 2.5 Flash61.9#466
7GPT-4.157.1#506
8Claude 3.7 Sonnet57.1#455
9MiniMax-M2.157.1#582
10Llama 4 Maverick54.8#1113
11QuanBench+ DeepSeek-Chat (checkpoint unspecified)42.9
12Qwen 2.5 7B Instruct19#2732

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=quanbench-plus-feedback-repair-qiskit · How It Works · Data refreshed daily, snapshot 2026-11-22.