ObligBench - Treatment - Intention Planning - Oracle Return Boolean: leaderboard
ObligBench frozen treatment cohort, strategy=p3-intention-planning, native metric=Oracle Return/boolean. Native assertion-mediated return-subcategory score_sum/scorable obligations among executable rows with passed assertion-mutation evidence. The primary analyzer suppresses subcategory cells with fewer than10 scorable obligations; suppressed and zero-denominator cells are absent, not zero. Object-return unobservability remains a diagnostic. Conditional task/obligation populations differ by model's generated test; each exact numerator, denominator and eligibility count is retained, not treated as a common all-task success rate. No pooled BVA/CFA/oracle aggregate, representability capability score, best-strategy selection or extra missing-data penalty is invented. Execution failure is accounted in the separate all-task execution source. Exact FP8 checkpoints and source-selected run/temperature mixtures remain distinct: GPT5.4 planning/feedback mix0/0.2; Panta temperature unreported. Release ab90ce896c162bcce9ccc677b6019a8dee05f974. Metric: Conditional native adequacy fraction (0-1).
Metric: Conditional native adequacy fraction (0-1). Source: arxiv.org. Saturation forecast: Around January 2027. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.4 | 0.96 | #82 |
| 2 | Gemini 3.6 Flash | 0.95 | #57 |
No result here: #3 Claude Opus 5.5, #5 GPT-6 Astra, #6 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=obligbench-treatment-intention-planning-oracle-return-boolean · How It Works · Data refreshed daily, snapshot 2026-10-27.