SastBench: leaderboard
Agentic triage of static application security testing (SAST) findings (SastBench, Feiglin and Dar, 2026): for 2,737 entries that each group one CWE's Semgrep findings in one commit of a real repository, a plain ReAct agent with file-reading and code-search tools decides whether the entry is a true vulnerability (a CVE reported after February 2025) or a false positive. The score is the Matthews correlation coefficient between verdicts and labels, from -1 to 1 with 0 at chance; higher is better.
Metric: Matthews Correlation Coefficient (-1 to 1, 0 = chance; Simple ReAct agent, one run over 2,737 CWE-grouped findings). Source: arxiv.org. Saturation forecast: Around February 2027. 7 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4.5 | 0.1 | #244 |
| 2 | Gemini 2.5 Pro | 0.08 | #262 |
| 3 | GPT-OSS-120B | 0.08 | #699 |
| 4 | Gemini 2.5 Flash | 0.01 | #452 |
| 5 | Qwen 3 Coder 480B A35B Instruct | -0.01 | #641 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=sastbench · How It Works · Data refreshed daily, snapshot 2026-11-12.