MoveBench - Plan Generation published exact values: leaderboard

MoveBench (2610.26185v1, Figure 1) uses the exact printed values. Its 809 questions comprise 113 single-select, 197 multi-select, and 499 open-ended questions, with 5012 criteria. The domain counts are 257 program-generation, 291 data-analysis, and 261 QA questions. Multiple-choice answers use exact match. For open-ended answers, earned rubric points are pooled over possible points within each format. The overall score weights the three format scores by successfully scored question counts; domain scores separately average per-question normalized points. The judges are Gemini 3.1 Pro, Claude 4.6 Sonnet, and GPT 5.2, with weights .35/.35/.30. An item receives zero when 2 of 3 judges flag a safety violation. Failed requests are retried; unresolved failures are excluded, and per-model scored denominators are not supplied. Native model names retain the study prefix; source GPT 6 is not resolved to Astra or Luna. The public 80-question GitHub release includes rubrics, but is a distinct subset/version and does not establish that all 809 scored questions are released. Section 4.1 supplies exact domain examples for two models. Domain scores macro-average each question’s e_i/m_i under Equation 7; they remain distinct from the overall format-weighted score. Metric: Normalized score.

Metric: Normalized score. Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 2 models tracked.

Top models

Interactive version: theaggregate.ai/benchmark?slug=movebench-plan-generation-published-exact-values · How It Works · Data refreshed daily, snapshot 2026-10-28.