VulnGym (Hard) - Entry-Point Identification: leaderboard
An oracle-conditioned task of VulnGym (Tencent, CUHK, Fudan University, HKU and Peking University, 2026) on the Hard level, 31 advisories whose vulnerability traces have nine or more steps: given the critical operation, the coding agent must find the externally reachable entry points that lead to it. Line recall is the percentage of the 43 reference entry points located within five lines; higher is better.
Metric: Line recall (%; share of the 43 reference entry points the agent localizes, given the oracle critical operation; Hard level of VulnGym (reference traces of 9 or more steps: 31 GitHub advisories, 46 vulnerability entries); read-only shell commands; locations match within 5 lines of the same repository-relative path). Source: arxiv.org. Saturation forecast: Around December 2027. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GLM-5.2 | 18.6 | #138 |
| 2 | MiniMax-M3 | 16.28 | #269 |
| 3 | DeepSeek V4 Flash | 6.98 | #258 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=vulngym-hard-entry-point-identification · How It Works · Data refreshed daily, snapshot 2026-11-13.