AsyncPlan Robo Challenge (PDDL2.1 Formalizer): leaderboard
Robo Challenge from Robust Asynchronous Planning via Auto-Formalization (Zhang et al., 2026): 140 kitchen tasks built on Robotouille in seven splits, with object states, shared stations, resources and multiple agents. Here the LLM writes PDDL2.1 files that the OPTIC temporal planner solves, zero-shot at temperature 0; plan accuracy (a valid plan reaching the optimal makespan) in percent, higher is better.
Metric: Plan accuracy (%; share of the 140 Robo Challenge kitchen tasks (seven splits of 20 built on Robotouille, with object states, resources, stations and multiple agents) whose plan is valid and reaches the optimal makespan; the LLM writes PDDL2.1 domain and problem files that the OPTIC temporal planner solves (PDDL2.1 Formalizer); zero-shot, temperature 0). Source: arxiv.org. Saturation forecast: Around February 2027. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3.6 35B A3B | 67.9 | #461 |
| 2 | GPT-5 Mini | 55.7 | #349 |
| 3 | Gemini 3 Flash | 52.9 | #159 |
| 4 | DeepSeek V4 Flash | 27.9 | #254 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=asyncplan-robo-challenge-pddl2-1-formalizer · How It Works · Data refreshed daily, snapshot 2026-11-12.