Pyrite - Reasoner Curated 265 - In-Context: leaderboard
Pyrite - Reasoner Curated 265 - In-Context: Pairwise pose-ranking accuracy (0-1). Pyrite arXiv 2611.14511v1 Tables 2-3: pairwise pose-ranking accuracy (0-1), mean and 95% confidence interval. Post-AF3 curated 265 pairs and full 484 pairs remain separate. PyriteReasoner prompt/input ablations and PyriteFeaturizer code/LLM-subcall feature discovery plus pre-AF3-trained logistic regression remain separate native runs. Paper names four unmodified backbones; effort, decoding and exact dated API snapshots are not reported. GRPO-trained rows are held without a released checkpoint identity.
Metric: Pairwise pose-ranking accuracy (0-1). Source: arxiv.org. Saturation forecast: Around 2028. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.8 | 0.62 | #56 |
| 2 | Qwen 3.8 27B | 0.59 | #199 |
| 3 | Claude Sonnet 5 | 0.58 | #108 |
| 4 | GPT-5.6 Sol | 0.47 | #29 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=pyrite-reasoner-curated-265-in-context · How It Works · Data refreshed daily, snapshot 2026-11-12.