ObligBench - Treatment - Mutation Feedback - CFA Catch: leaderboard
ObligBench frozen treatment cohort, strategy=p5-mutation-feedback, native metric=CFA/catch. Native partial-credit score_sum/scorable obligations among executable rows, separately by control-flow construct; exact upstream represented-count fallback retained. This is not the distinct checked/applicable column called a pessimistic bound in the report. Conditional task/obligation populations differ by model's generated test; each exact numerator, denominator and eligibility count is retained, not treated as a common all-task success rate. No pooled BVA/CFA/oracle aggregate, representability capability score, best-strategy selection or extra missing-data penalty is invented. Execution failure is accounted in the separate all-task execution source. Exact FP8 checkpoints and source-selected run/temperature mixtures remain distinct: GPT5.4 planning/feedback mix0/0.2; Panta temperature unreported. Release ab90ce896c162bcce9ccc677b6019a8dee05f974. Metric: Conditional native adequacy fraction (0-1).
Metric: Conditional native adequacy fraction (0-1). Source: arxiv.org. Saturation forecast: Around 2029. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.6 Flash | 0.15 | #57 |
| 2 | GPT-5.4 | 0.13 | #82 |
No result here: #3 Claude Opus 5.5, #5 GPT-6 Astra, #6 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=obligbench-treatment-mutation-feedback-cfa-catch · How It Works · Data refreshed daily, snapshot 2026-10-27.