SCDBench (Compilation Repair): leaderboard
SCDBench (Qin, Song and Gervais, 2026) tests LLM decompilation of 600 verified Ethereum smart contracts in three difficulty tiers: from EVM assembly and function-selector hints the model must write Solidity that compiles and behaves like the original. Percentage of the 14,553 public functions whose replayed test cases all match the original after the model gets one attempt to fix the compiler errors of its first answer; higher is better.
Metric: Semantically consistent public functions (%; all replayed test cases match return data, revert status, logs and storage, of 14,553 functions in 600 contracts, after one same-model repair attempt from compiler errors). Source: arxiv.org. Saturation forecast: Around April 2028. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.7 (High) | 29.44 | #75 (Claude Opus 4.7) |
| 2 | GPT-5.3 Codex (High) | 28.02 | #114 (GPT-5.3 Codex) |
| 3 | GLM-5 (Thinking) | 20.72 | #256 (GLM-5) |
| 4 | GLM-5 (Non-reasoning) | 17.66 | #256 (GLM-5) |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=scdbench-compilation-repair · How It Works · Data refreshed daily, snapshot 2026-11-12.