Reconstruction (Research Idea Recovery) - Astronomy: leaderboard
Metric: Match rate (%; the 85 astronomy seed papers; share of a model's five hypotheses, proposed in one call from a published paper's pre-publication bibliography only (titles and abstracts under anonymous IDs, the paper itself and later literature withheld), that an LLM judge matches to the held-out paper's idea; mean over the six other evaluated models as leave-one-out judges). Source: arxiv.org. Saturation forecast: Around 2029. 7 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | Claude Opus 4.8 | 14.7 |
| 2 | GPT-5.6 Pro Sol | 12.9 |
| 3 | Kimi K3 | 10.7 |
| 4 | GLM-5.2 | 9.5 |
| 5 | Gemini 3.1 Pro (Preview) | 8.9 |
| 6 | DeepSeek V4 Pro | 6.7 |
| 7 | Qwen 3.7 Max | 5.3 |
Interactive version: theaggregate.ai/benchmark?slug=reconstruction-research-idea-recovery-astronomy · How It Works · Data refreshed daily, snapshot 2026-09-26.