SCDBench (Compilation Repair) - ABI Recovery: leaderboard
SCDBench (Qin, Song and Gervais, 2026) tests LLM decompilation of 600 verified Ethereum smart contracts given as EVM assembly with function-selector hints. This board scores interface recovery after the model gets one attempt to repair its output from the compiler errors: micro F1 in percent of the recovered public function signatures against the ground-truth ABI; higher is better.
Metric: ABI recovery F1 (%; micro F1 of the recovered public function signatures against the ground-truth ABI over all 600 contracts, extra functions counting as false positives and missing ones as false negatives, printed as a fraction and scaled to percent; after one same-model repair attempt from the compiler errors). Source: arxiv.org. Saturation forecast: Around February 2027. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.3 Codex (High) | 89.6 | #116 (GPT-5.3 Codex) |
| 2 | Claude Opus 4.7 (High) | 72.1 | #79 (Claude Opus 4.7) |
| 3 | GLM-5 (Thinking) | 69.7 | #260 (GLM-5) |
| 4 | GLM-5 (Non-reasoning) | 65.8 | #260 (GLM-5) |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=scdbench-compilation-repair-abi-recovery · How It Works · Data refreshed daily, snapshot 2026-11-13.