AdaPlanBench - Valid Plan Rate: leaderboard
AdaPlanBench (2026) gives LLM agents 307 household planning tasks whose world and user constraints stay hidden until a proposed plan violates one, so the agent must re-plan from feedback; boards use the mid-level constraint profile. This board is the percentage of tasks that end with a plan satisfying every constraint, before the turn limit and without an early stop; higher is better.
Metric: Valid plan rate (%): share of tasks that end with a plan satisfying every constraint rather than an early stop or the turn limit, without the rubric gate, 307 household planning tasks with progressively revealed world and user constraints at the mid-level constraint profile: a hidden constraint is disclosed only when a proposed plan violates it (GPT-5.4 constraint judges), and the agent re-plans within the turn budget; temperature 0 (GPT-5 family at its fixed default, mean of three runs); higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 10 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) | 91.21 | #94 |
| 2 | Gemini 3 Flash | 90.23 | #164 |
| 3 | GPT-5 | 89.58 | #167 |
| 4 | GPT-5 Mini | 85.34 | #348 |
| 5 | Llama 3.3 70B Instruct | 83.71 | #1357 |
| 6 | Qwen 3 8B | 82.35 | #1830 |
| 7 | Qwen 3 32B | 80.13 | #960 |
| 8 | DeepSeek V4 Flash | 76.97 | #258 |
| 9 | Qwen 3 14B | 73.62 | #1233 |
| 10 | GPT-5 Nano | 67.75 | #889 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=adaplanbench-valid-plan-rate · How It Works · Data refreshed daily, snapshot 2026-11-13.