QuanBench+ (Pass@5) - PennyLane: leaderboard

Quantum code generation in PennyLane on QuanBench+ (2026) with five samples per task at temperature 0.8: a task counts as solved when any of the five programs passes the executable functional tests of the 42 framework-aligned tasks. Pass@5 in percent; higher is better.

Metric: Pass@5 (%): any of five PennyLane programs sampled at temperature 0.8 passes, on the 42 framework-aligned QuanBench+ tasks (quantum algorithms, gate decomposition, state preparation), graded by executable functional tests with KL-divergence acceptance for probabilistic outputs; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 12 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1DeepSeek R159.5#482
2GPT-5.157.1#240
3MiniMax-M2.154.8#575
4Kimi K2 (Thinking)52.4
5GLM-4.747.6#380
6QuanBench+ DeepSeek-Chat (checkpoint unspecified)47.6
7Gemini 3 Pro40.5#145
8Claude 3.7 Sonnet38.1#470
9Gemini 2.5 Flash35.7#463
10GPT-4.135.7#505
11Llama 4 Maverick31#1110
12Qwen 2.5 7B Instruct21.4#2723

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=quanbench-plus-pass-5-pennylane · How It Works · Data refreshed daily, snapshot 2026-11-21.