Pyrite - Featurizer Curated 265 - Pose LLMSubcall: leaderboard
Pyrite - Featurizer Curated 265 - Pose LLMSubcall: Pairwise pose-ranking accuracy (0-1). Pyrite arXiv 2611.14511v1 Tables 2-3: pairwise pose-ranking accuracy (0-1), mean and 95% confidence interval. Post-AF3 curated 265 pairs and full 484 pairs remain separate. PyriteReasoner prompt/input ablations and PyriteFeaturizer code/LLM-subcall feature discovery plus pre-AF3-trained logistic regression remain separate native runs. Paper names four unmodified backbones; effort, decoding and exact dated API snapshots are not reported. GRPO-trained rows are held without a released checkpoint identity.
Metric: Pairwise pose-ranking accuracy (0-1). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 5 (Benchmark Sweep Run: source=Pyrite; harness=PyriteFeaturizer; input=Pose LLMSubcall; effort=unreported) | 0.73 | |
| 2 | GPT-5.6-sol (Benchmark Sweep Run: source=Pyrite; harness=PyriteFeaturizer; input=Pose LLMSubcall; effort=unreported) | 0.71 | |
| 3 | Claude Opus 4.8 (Benchmark Sweep Run: source=Pyrite; harness=PyriteFeaturizer; input=Pose LLMSubcall; effort=unreported) | 0.69 | |
| 4 | Qwen3.8-27B (Benchmark Sweep Run: source=Pyrite; harness=PyriteFeaturizer; input=Pose LLMSubcall; effort=unreported) | 0.59 |
Interactive version: theaggregate.ai/benchmark?slug=pyrite-featurizer-curated-265-pose-llmsubcall · How It Works · Data refreshed daily, snapshot 2026-11-12.