ObligBench - Control - Zero-Shot - CFA Case: leaderboard

ObligBench frozen control cohort, strategy=zero-shot, native metric=CFA/case. Native partial-credit score_sum/scorable obligations among executable rows, separately by control-flow construct; exact upstream represented-count fallback retained. This is not the distinct checked/applicable column called a pessimistic bound in the report. Conditional task/obligation populations differ by model's generated test; each exact numerator, denominator and eligibility count is retained, not treated as a common all-task success rate. No pooled BVA/CFA/oracle aggregate, representability capability score, best-strategy selection or extra missing-data penalty is invented. Execution failure is accounted in the separate all-task execution source. Exact FP8 checkpoints and source-selected run/temperature mixtures remain distinct: GPT5.4 planning/feedback mix0/0.2; Panta temperature unreported. Release ab90ce896c162bcce9ccc677b6019a8dee05f974. Metric: Conditional native adequacy fraction (0-1).

Metric: Conditional native adequacy fraction (0-1). Source: arxiv.org. Saturation forecast: Around January 2027. 2 models tracked.

Top models

No result here: #3 Claude Opus 5.5, #5 GPT-6 Astra, #6 GPT-6.1 Sol.

Interactive version: theaggregate.ai/benchmark?slug=obligbench-control-zero-shot-cfa-case · How It Works · Data refreshed daily, snapshot 2026-10-27.