AsyncPlan-XXL (CP-SAT Formalizer): leaderboard
AsyncPlan-XXL (Zhang et al., 2026) scales asynchronous scheduling to dependency graphs of 5 to 100 actions with durations and precedence constraints, 50 instances per size. Here the LLM writes a CP-SAT constraint program that OR-Tools solves, zero-shot at temperature 0; plan accuracy (a valid schedule that reaches the optimal makespan) in percent averaged over the 12 sizes, higher is better.
Metric: Plan accuracy (%; mean over the 12 AsyncPlan-XXL graph sizes (5 to 100 actions, 50 instances each) of the share of schedules that satisfy every duration and precedence constraint and reach the optimal makespan; the LLM writes a CP-SAT constraint program that OR-Tools solves (CP-SAT Formalizer); zero-shot, temperature 0). Source: arxiv.org. Saturation forecast: Estimated already saturated. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3 Flash | 98 | #159 |
| 2 | DeepSeek V4 Flash | 97 | #254 |
| 3 | Qwen 3.6 35B A3B | 96 | #461 |
| 4 | GPT-5 Mini | 86 | #349 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=asyncplan-xxl-cp-sat-formalizer · How It Works · Data refreshed daily, snapshot 2026-11-12.