Physics Expert Audit - CMT-Benchmark - pass@4: leaderboard
Expert-validated and repaired physics questions, with the published pass@4 scoring protocol. Distinct corrected population; public-source tasks use no tools, expert tasks use Claude Code/Codex for Fable/Sol and no tools for Gemini. Not fused into the original benchmark.
Metric: Validated/repaired accuracy (%, pass@4). Source: arxiv.org. Saturation forecast: Estimated already saturated. 3 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.6 Sol (High) | 97.96 | #18 (GPT-5.6 Sol) |
| 2 | Claude Fable 5 (High) | 93.88 | #15 (Claude Fable 5) |
| 3 | Gemini 3.1 Pro (Preview) (High) | 87.76 | #56 (Gemini 3.1 Pro (Preview)) |
No result here: #3 Claude Opus 5.5, #5 GPT-6 Astra, #7 Gemini 4 Argon.
Interactive version: theaggregate.ai/benchmark?slug=physics-expert-audit-cmt-benchmark-pass-4 · How It Works · Data refreshed daily, snapshot 2026-10-16.