PatchEval - Location Oracle: leaderboard
Patch real-world CVEs in Go, JavaScript and Python when told which functions are vulnerable, with no access to the proof-of-concept exploit or unit-test results. A patch counts only if it defeats the exploit and keeps the unit tests passing. This is the original PatchEval harness, not PatchEval-Verified. Plain model runs and agent runs (SWE-agent, OpenHands, Claude Code) are both ranked; the agent is named in the label. Higher is better. Source: patcheval.github.io.
Metric: % POC & Unit. Source: patcheval.github.io. Saturation forecast: Around 2029. 16 models tracked. Updated 3 Oct 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | GPT-5 | 29.13 | #177 | |
| 2 | Gemini 2.5 Pro | 22.61 | #274 | |
| 3 | DeepSeek V3.2 Speciale | 21.74 | #272 | |
| 4 | GPT-4.1 | 21.3 | #505 | |
| 5 | DeepSeek R1 | 19.57 | #487 | |
| 6 | DeepSeek R1 0528 | 19.13 | #431 | |
| 7 | GPT-4.1 (2025-04-14) | 18.7 | #501 | |
| 8 | DeepSeek V3.2 | Thinking | 18.7 | #417 |
| 9 | DeepSeek V3 (0324) | 17.83 | #722 | |
| 10 | Qwen 3 Max | 17.83 | #413 | |
| 11 | Qwen 3 Coder 480B A35B Instruct | 17.83 | #653 | |
| 12 | O3 (2025-04-16) | 17.39 | #229 | |
| 13 | Doubao-Seed-1.8 (2025-12-15) | 17.39 | ||
| 14 | Kimi K2 | 16.52 | #446 | |
| 15 | Doubao-Seed-1.6 (2025-06-15) | 16.52 |
No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.
Interactive version: theaggregate.ai/benchmark?slug=patcheval-location-oracle · How It Works · Data refreshed daily, snapshot 2026-11-17.