Pyrite - Featurizer Curated 265 - Code Only: leaderboard
Pyrite - Featurizer Curated 265 - Code Only: Pairwise pose-ranking accuracy (0-1). Pyrite arXiv 2611.14511v1 Tables 2-3: pairwise pose-ranking accuracy (0-1), mean and 95% confidence interval. Post-AF3 curated 265 pairs and full 484 pairs remain separate. PyriteReasoner prompt/input ablations and PyriteFeaturizer code/LLM-subcall feature discovery plus pre-AF3-trained logistic regression remain separate native runs. Paper names four unmodified backbones; effort, decoding and exact dated API snapshots are not reported. GRPO-trained rows are held without a released checkpoint identity.
Metric: Pairwise pose-ranking accuracy (0-1). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.8 (Benchmark Sweep Run: source=Pyrite; harness=PyriteFeaturizer; input=Code Only; effort=unreported) | 0.65 | |
| 2 | GPT-5.6-sol (Benchmark Sweep Run: source=Pyrite; harness=PyriteFeaturizer; input=Code Only; effort=unreported) | 0.65 | |
| 3 | Claude Sonnet 5 (Benchmark Sweep Run: source=Pyrite; harness=PyriteFeaturizer; input=Code Only; effort=unreported) | 0.63 | |
| 4 | Qwen3.8-27B (Benchmark Sweep Run: source=Pyrite; harness=PyriteFeaturizer; input=Code Only; effort=unreported) | 0.52 |
Interactive version: theaggregate.ai/benchmark?slug=pyrite-featurizer-curated-265-code-only · How It Works · Data refreshed daily, snapshot 2026-11-12.