NLCO - Solver-Calling Code (Set-L) - Accuracy: leaderboard
Share of large (Set-L) NLCO instances solved exactly when the model writes Gurobi solver code instead of answering directly; the solution counts only if it is feasible and optimal. Percent, higher is better. NLCO (Jiang et al., Eindhoven University of Technology and others, 2026) renders 43 combinatorial optimization problem types (routing, packing, scheduling, covering, assignment) as natural-language scenarios with solver-annotated reference solutions.
Metric: Accuracy of generated Gurobi solver code: feasible and optimal (%). Source: arxiv.org. Saturation forecast: Around February 2027. 16 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3 Flash (High) | 83.77 | #159 (Gemini 3 Flash) |
| 2 | Claude Sonnet 4.5 (Thinking) | 80.9 | #244 (Claude Sonnet 4.5) |
| 3 | GPT-5.1 (Medium) | 76.88 | #225 (GPT-5.1) |
| 4 | DeepSeek V3.2 (Thinking) | 76.19 | #403 (DeepSeek V3.2) |
| 5 | Qwen 3 235B A22B 2507 Instruct | 69.26 | #619 |
| 6 | Grok 4.1 Fast (Reasoning) | 67.26 | #426 (Grok 4.1 Fast) |
| 7 | DeepSeek V3.2 (Non-reasoning) | 66.98 | #403 (DeepSeek V3.2) |
| 8 | O4 Mini (High) | 56.47 | #354 (O4 Mini) |
| 9 | QwQ-32B | 54.7 | #909 |
| 10 | Llama 4 Maverick Instruct | 53.07 | #1125 |
| 11 | Qwen 3 14B (Reasoning) | 51.21 | #1235 (Qwen 3 14B) |
| 12 | MiMo-V2-Flash (Non-reasoning) | 41.72 | #674 (MiMo-V2-Flash) |
| 13 | nemotron-3-nano-30B-a3B (Reasoning) | 41.44 | #1891 (nemotron-3-nano-30B-a3B) |
| 14 | Ministral-3-14B-Instruct-2512 | 40.6 | #1595 |
| 15 | Qwen 3 14B (Non-reasoning) | 39.58 | #1235 (Qwen 3 14B) |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=nlco-solver-calling-code-set-l-accuracy · How It Works · Data refreshed daily, snapshot 2026-11-12.