MM-OptBench (Pass@4): leaderboard

MM-OptBench (2026) asks multimodal models to turn a text specification plus visual artifacts such as tables, graphs, maps and schedules into an optimization formulation and solver code, which is run and checked against the verified optimum. This board is pass@4 in percent over all 780 instances, counting an instance solved if any of four sampled runs is correct; higher is better.

Metric: pass@4 (%) over the 780 instances, success in at least one of four sampled runs, solver-grounded: the model reads a text specification plus visual artifacts (tables, graphs, maps, schedules) and must return an optimization formulation and executable solver code whose returned objective matches the verified optimum, prompting only with no fine-tuning, retrieval or agentic repair; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 7 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1GPT-5.468.3#135
2Gemini 3.1 Pro (Preview)64.4#99
3Qwen 3 VL Plus34.7#512
4Claude Sonnet 4.631.3#173
5GLM-4.5V29.1#754
6Qwen-VL-Max (MM-OptBench checkpoint unspecified)25.5
7MM-Eureka-Qwen-7B0

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=mm-optbench-pass-4 · How It Works · Data refreshed daily, snapshot 2026-11-21.