PatchBench: leaderboard
PatchBench (2026) evaluates AI coding agents on 213 C and C++ vulnerability-patching tasks from 32 GitHub projects, with transplanted and mutated vulnerabilities so that memorized historical fixes do not apply. A task is solved when the patch stops the original crash and passes further security checks and semantic checks such as unit tests and output-state comparison. Solved rate in percent under a fixed per-task budget; higher is better.
Metric: Solved rate (%; share of 213 C/C++ repository-level vulnerability-patching tasks from 32 GitHub projects, 16 CWE types, made memorization-resistant by vulnerability transplant and code mutation, whose patch stops the original PoC crash and passes security validation (additional PoCs, sanitizer regression) and semantic validation (unit tests, output-state check); 5 US dollar budget per task, web access disabled, medium reasoning effort). Source: arxiv.org. Saturation forecast: Around March 2027. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.6 Sol (Medium) | 58.2 | #29 (GPT-5.6 Sol) |
| 2 | Claude Opus 4.8 (Medium) | 47.9 | #56 (Claude Opus 4.8) |
| 3 | Gemini 3.5 Flash (Medium) | 44.1 | #98 (Gemini 3.5 Flash) |
| 4 | GPT-5 (Medium) | 42.3 | #163 (GPT-5) |
| 5 | Gemini 3.1 Pro (Preview) (Medium) | 31.5 | #91 (Gemini 3.1 Pro (Preview)) |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=patchbench · How It Works · Data refreshed daily, snapshot 2026-11-12.