HoF-Bench: leaderboard
HoF-Bench (AISLE, 2026) asks LLM vulnerability scanners to rediscover 95 public CVEs that AISLE found in eight open-source repositories such as OpenSSL and curl, running each model in one minimal scanner scaffold with four passes and multi-round triage; a detector-blinded judge credits a finding only when code path, root cause, attack condition and impact all match. Score is the number of the 95 CVEs rediscovered in any of the four passes, with the optional generated-context stage enabled; higher is better.
Metric: CVEs rediscovered at pass@4 (out of 95), with the generated-context stage, on 95 public AI-discovered CVEs in eight repositories pinned at vulnerable commits, found by a minimal fixed LLM scanner scaffold with four repeated passes and multi-round triage, credited only when a detector-blinded GPT-5.6 terra judge matches code path, root cause, attack condition and impact; medium reasoning effort where supported. Source: arxiv.org. Saturation forecast: Around February 2027. 10 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.6 Luna (Medium) | 65 | #121 (GPT-5.6 Luna) |
| 2 | DeepSeek V4 Flash | 62 | #258 |
| 3 | GPT-5.4 Mini (Medium) | 58 | #389 (GPT-5.4 Mini) |
| 4 | Gemini 3.5 Flash (Medium) | 56 | #101 (Gemini 3.5 Flash) |
| 5 | GPT-OSS-120B | 52 | #700 |
| 6 | Qwen 3.6 35B A3B | 50 | #462 |
| 7 | Gemini 2.5 Flash (Medium) | 47 | #453 (Gemini 2.5 Flash) |
| 8 | GPT-5.4 Nano (Medium) | 46 | #635 (GPT-5.4 Nano) |
| 9 | GPT-OSS-20B | 40 | #1147 |
| 10 | Mistral Small 4 | 38 | #1201 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=hof-bench · How It Works · Data refreshed daily, snapshot 2026-11-13.