NLCO - Set-M - Accuracy: leaderboard
Share of medium (Set-M) NLCO instances, 50 per task, where the model's own discrete answer is feasible and reaches the solver's optimal objective within tolerance; no code or external solver is allowed. Percent, higher is better. NLCO (Jiang et al., Eindhoven University of Technology and others, 2026) renders 43 combinatorial optimization problem types (routing, packing, scheduling, covering, assignment) as natural-language scenarios with solver-annotated reference solutions.
Metric: Accuracy: feasible and matching the solver-optimal objective (%). Source: arxiv.org. Saturation forecast: Around February 2027. 16 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3 Flash (High) | 76.8 | #159 (Gemini 3 Flash) |
| 2 | GPT-5.1 (Medium) | 72.8 | #225 (GPT-5.1) |
| 3 | Grok 4.1 Fast (Reasoning) | 70.9 | #426 (Grok 4.1 Fast) |
| 4 | O4 Mini (High) | 70.7 | #354 (O4 Mini) |
| 5 | Claude Sonnet 4.5 (Thinking) | 70.4 | #244 (Claude Sonnet 4.5) |
| 6 | DeepSeek V3.2 (Thinking) | 69.1 | #403 (DeepSeek V3.2) |
| 7 | Qwen 3 235B A22B 2507 Instruct | 54.9 | #619 |
| 8 | nemotron-3-nano-30B-a3B (Reasoning) | 54.2 | #1891 (nemotron-3-nano-30B-a3B) |
| 9 | DeepSeek V3.2 (Non-reasoning) | 47.7 | #403 (DeepSeek V3.2) |
| 10 | Qwen 3 14B (Reasoning) | 42.4 | #1235 (Qwen 3 14B) |
| 11 | QwQ-32B | 39.1 | #909 |
| 12 | MiMo-V2-Flash (Non-reasoning) | 37.4 | #674 (MiMo-V2-Flash) |
| 13 | Ministral-3-14B-Instruct-2512 | 25.4 | #1595 |
| 14 | Llama 4 Maverick Instruct | 19 | #1125 |
| 15 | Qwen 3 14B (Non-reasoning) | 17.5 | #1235 (Qwen 3 14B) |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=nlco-set-m-accuracy · How It Works · Data refreshed daily, snapshot 2026-11-12.