PatchEval - Location Oracle: leaderboard

Patch real-world CVEs in Go, JavaScript and Python when told which functions are vulnerable, with no access to the proof-of-concept exploit or unit-test results. A patch counts only if it defeats the exploit and keeps the unit tests passing. This is the original PatchEval harness, not PatchEval-Verified. Plain model runs and agent runs (SWE-agent, OpenHands, Claude Code) are both ranked; the agent is named in the label. Higher is better. Source: patcheval.github.io.

Metric: % POC & Unit. Source: patcheval.github.io. Saturation forecast: Around 2029. 16 models tracked. Updated 3 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-529.13#177
2Gemini 2.5 Pro22.61#274
3DeepSeek V3.2 Speciale21.74#272
4GPT-4.121.3#505
5DeepSeek R119.57#487
6DeepSeek R1 052819.13#431
7GPT-4.1 (2025-04-14)18.7#501
8DeepSeek V3.2Thinking18.7#417
9DeepSeek V3 (0324)17.83#722
10Qwen 3 Max17.83#413
11Qwen 3 Coder 480B A35B Instruct17.83#653
12O3 (2025-04-16)17.39#229
13Doubao-Seed-1.8 (2025-12-15)17.39
14Kimi K216.52#446
15Doubao-Seed-1.6 (2025-06-15)16.52

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=patcheval-location-oracle · How It Works · Data refreshed daily, snapshot 2026-11-17.