MoveBench - Open Questions published exact values: leaderboard
MoveBench (2610.26185v1, Figure 1) uses the exact printed values. Its 809 questions comprise 113 single-select, 197 multi-select, and 499 open-ended questions, with 5012 criteria. The domain counts are 257 program-generation, 291 data-analysis, and 261 QA questions. Multiple-choice answers use exact match. For open-ended answers, earned rubric points are pooled over possible points within each format. The overall score weights the three format scores by successfully scored question counts; domain scores separately average per-question normalized points. The judges are Gemini 3.1 Pro, Claude 4.6 Sonnet, and GPT 5.2, with weights .35/.35/.30. An item receives zero when 2 of 3 judges flag a safety violation. Failed requests are retried; unresolved failures are excluded, and per-model scored denominators are not supplied. Native model names retain the study prefix; source GPT 6 is not resolved to Astra or Luna. The public 80-question GitHub release includes rubrics, but is a distinct subset/version and does not establish that all 809 scored questions are released. Section 4.2 supplies eleven exact numeric values in prose; the other four values are unlabeled in the plot. The open-question score sums earned positive-rubric points over available possible points, with the safety-vote rule applied to zero individual items. Metric: Normalized score.
Metric: Normalized score. Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 11 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | MoveBench study Claude 4.6 Opus | 0.92 | |
| 2 | MoveBench study GPT-5.2 | 0.9 | |
| 3 | MoveBench study GPT-5.5 | 0.89 | |
| 4 | MoveBench study Qwen3.5-27B | 0.89 | |
| 5 | MoveBench study Doubao Seed 2.0 Pro | 0.89 | |
| 6 | MoveBench study Qwen3 Max | 0.89 | |
| 7 | MoveBench study GLM-5.3 | 0.87 | |
| 8 | MoveBench study Qwen3.8 Max | 0.86 | |
| 9 | MoveBench study GPT-6 | 0.84 | |
| 10 | MoveBench study Hy3 | 0.83 |
Interactive version: theaggregate.ai/benchmark?slug=movebench-open-questions-published-exact-values · How It Works · Data refreshed daily, snapshot 2026-10-28.