SCDBench (Compilation Repair) - ABI Recovery: leaderboard
Metric: ABI recovery F1 (%; micro F1 of the recovered public function signatures against the ground-truth ABI over all 600 contracts, extra functions counting as false positives and missing ones as false negatives, printed as a fraction and scaled to percent; after one same-model repair attempt from the compiler errors). Source: arxiv.org. Saturation forecast: Around December 2026. 4 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | GPT-5.3 Codex (High) | 89.6 |
| 2 | Claude Opus 4.7 (High) | 72.1 |
| 3 | GLM-5 (Thinking) | 69.7 |
| 4 | GLM-5 (Non-reasoning) | 65.8 |
Interactive version: theaggregate.ai/benchmark?slug=scdbench-compilation-repair-abi-recovery · How It Works · Data refreshed daily, snapshot 2026-09-29.