MoveBench - Multi Select published exact values: leaderboard

MoveBench (2610.26185v1, Figure 1) uses the exact printed values. Its 809 questions comprise 113 single-select, 197 multi-select, and 499 open-ended questions, with 5012 criteria. The domain counts are 257 program-generation, 291 data-analysis, and 261 QA questions. Multiple-choice answers use exact match. For open-ended answers, earned rubric points are pooled over possible points within each format. The overall score weights the three format scores by successfully scored question counts; domain scores separately average per-question normalized points. The judges are Gemini 3.1 Pro, Claude 4.6 Sonnet, and GPT 5.2, with weights .35/.35/.30. An item receives zero when 2 of 3 judges flag a safety violation. Failed requests are retried; unresolved failures are excluded, and per-model scored denominators are not supplied. Native model names retain the study prefix; source GPT 6 is not resolved to Astra or Luna. The public 80-question GitHub release includes rubrics, but is a distinct subset/version and does not establish that all 809 scored questions are released. Section 4.2 supplies eight exact numeric values in prose. Five approximate values and two unreported values are not reconstructed from unlabeled bar heights. This is the available source score subset, not a complete 15-model cohort. Metric: Normalized score.

Metric: Normalized score. Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 8 models tracked.

Top models

Interactive version: theaggregate.ai/benchmark?slug=movebench-multi-select-published-exact-values · How It Works · Data refreshed daily, snapshot 2026-10-28.