SemanticAlign-Bench - Numerical Precision: leaderboard
Metric: Semantic alignment score (0-1; numerical-precision claims (hyperparameters, thresholds, architectural dimensions); 30 ICLR, ICML and NeurIPS 2025 papers with 1,491 atomic implementation claims; the agent reads the paper and writes a reproduction repository, and GPT-5.5 scores each claim against the code on a five-level rubric (0, 0.25, 0.5, 0.75, 1) without executing it; mean over the 30 paper-level scores). Source: arxiv.org. Saturation forecast: Around December 2026. 12 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | Claude Sonnet 4.6 | 0.38 |
| 2 | DeepSeek V4 Pro | 0.36 |
| 3 | Gemini 2.5 Flash | 0.34 |
| 4 | GPT-4o | 0.08 |
Interactive version: theaggregate.ai/benchmark?slug=semanticalign-bench-numerical-precision · How It Works · Data refreshed daily, snapshot 2026-09-29.