MIPLIB-NL: leaderboard
Metric: Pass@1 accuracy (%), one sample per instance (MIPLIB-NL: natural-language optimization-modeling problems reverse-generated one-to-one from MIPLIB 2017 mixed-integer programs (223 instances, the 8 infeasible and 15 open ones excluded from scoring), with model and data specified separately; the model writes the formulation and solver code, which are executed; an instance is solved when the solver objective matches the certified optimum within 1e-6 relative error; temperature 0.6; direct prompting, no tools); higher is better. Source: arxiv.org. Saturation forecast: Around September 2027. 14 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.1 | 39.05 | #131 |
| 2 | Gemini 3 Pro (Preview) | 36.19 | #64 |
| 3 | Claude Sonnet 4.5 (Thinking) | 30 | #138 (Claude Sonnet 4.5) |
| 4 | Claude Sonnet 4.5 | 24.23 | #138 |
| 5 | DeepSeek V3.2 (Non-reasoning) | 23.81 | #198 (DeepSeek V3.2) |
| 6 | GPT-5.1 Codex | 23.33 | #154 |
| 7 | DeepSeek V3.2 (Thinking) | 22.38 | #198 (DeepSeek V3.2) |
| 8 | Qwen 3 Max (Preview) (Thinking) | 21.43 | #224 (Qwen 3 Max (Preview)) |
| 9 | Qwen 3 Max (Non-reasoning) | 21.43 | #201 (Qwen 3 Max) |
No result here: #3 Claude Opus 5.5, #5 GPT-6 Astra, #8 Claude Fable 5.1.
Interactive version: theaggregate.ai/benchmark?slug=miplib-nl · How It Works · Data refreshed daily, snapshot 2026-10-11.