Skepthical Study - Astrophysics Mathematics: leaderboard
Skepthical tests models and review pipelines on deliberately flawed research papers. Astrophysics and machine-learning results use separate 15-paper cohorts and expert error annotations. Free-form, structured and concise review prompts remain distinct. False-positive rates use all raised flags on the clean-paper controls. This board reports error detection rate (%). Higher is better.
Metric: Error detection rate (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 7 models tracked. Updated 14 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | GPT-5.5 | Evaluation run: Skepthical Study Concise Structured | 90.5 | #52 |
| 2 | Skepthical | Evaluation run: multi-provider pipeline frozen 2026-04-21 | 73.8 | |
| 3 | Claude Opus 4.7 | Evaluation run: Skepthical Study Claude Code Structured | 71.4 | #78 |
| 4 | Stanford Agentic Reviewer | Evaluation run: hosted reviewer 2026-05-01 | 50 | |
| 5 | Gemini 3.1 Pro Preview | Evaluation run: Skepthical Study Structured | 45.2 | |
| 6 | PaperAudit_Qwen3_14B_sft_rl | Evaluation run: Skepthical Study Deep | 2.4 | |
| 7 | Llama-OpenReviewer-8B | Evaluation run: Skepthical Study ICLR Template | 0 |
Interactive version: theaggregate.ai/benchmark?slug=skepthical-study-astrophysics-mathematics · How It Works · Data refreshed daily, snapshot 2026-11-16.