SemanticAlign-Bench: leaderboard
SemanticAlign-Bench (SA-Bench; Beihang University, Shanghai Jiao Tong University and Peking University, 2026) measures semantic drift in LLM-generated paper reproductions: an agent reads one of 30 ICLR, ICML and NeurIPS 2025 papers and writes a repository, and GPT-5.5 scores 1,491 atomic implementation claims against the code on a five-level rubric without executing it. The score from 0 to 1 is the mean of the numerical, method, protocol and ordering claim scores; higher is better.
Metric: Semantic alignment score (0-1; mean of the four drift-type scores; 30 ICLR, ICML and NeurIPS 2025 papers with 1,491 atomic implementation claims; the agent reads the paper and writes a reproduction repository, and GPT-5.5 scores each claim against the code on a five-level rubric (0, 0.25, 0.5, 0.75, 1) without executing it; mean over the 30 paper-level scores). Source: arxiv.org. Saturation forecast: Around March 2027. 12 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | DeepSeek V4 Pro | 0.28 | #188 |
| 2 | Claude Sonnet 4.6 | 0.28 | #161 |
| 3 | Gemini 2.5 Flash | 0.24 | #453 |
| 4 | GPT-4o | 0.08 | #741 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=semanticalign-bench · How It Works · Data refreshed daily, snapshot 2026-11-13.