QuanBench+ (Feedback Repair) - PennyLane: leaderboard

Quantum code generation in PennyLane with feedback repair on QuanBench+ (2026): after a runtime error or a wrong answer on one of the 42 framework-aligned tasks, the model receives the exception trace or its failing function and may submit a corrected program, up to five times. Pass@1 after repair, in percent; higher is better.

Metric: Pass@1 after feedback repair (%): after a runtime error the model receives the exception trace, after a wrong answer its failing function, and returns a corrected PennyLane program, up to five repair attempts per task, on the 42 framework-aligned QuanBench+ tasks (quantum algorithms, gate decomposition, state preparation), graded by executable functional tests with KL-divergence acceptance for probabilistic outputs; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 12 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 3 Pro66.7#145
2GPT-5.166.7#240
3GLM-4.764.3#380
4DeepSeek R152.4#482
5Claude 3.7 Sonnet47.6#470
6MiniMax-M2.147.6#575
7GPT-4.145.2#505
8QuanBench+ DeepSeek-Chat (checkpoint unspecified)45.2
9Gemini 2.5 Flash40.5#463
10Llama 4 Maverick38.1#1110
11Kimi K2 (Thinking)38.1
12Qwen 2.5 7B Instruct19#2723

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=quanbench-plus-feedback-repair-pennylane · How It Works · Data refreshed daily, snapshot 2026-11-21.