NLCO - Set-S - Feasibility Rate: leaderboard
Share of small (Set-S) NLCO instances, 50 per task, where the model's own discrete answer parses and satisfies every constraint, whatever its objective value; no code or external solver is allowed. Percent, higher is better. NLCO (Jiang et al., Eindhoven University of Technology and others, 2026) renders 43 combinatorial optimization problem types (routing, packing, scheduling, covering, assignment) as natural-language scenarios with solver-annotated reference solutions.
Metric: Average feasibility rate: output satisfies every constraint (%). Source: arxiv.org. Saturation forecast: Estimated already saturated. 16 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | O4 Mini (High) | 98.9 | #354 (O4 Mini) |
| 2 | GPT-5.1 (Medium) | 98.9 | #225 (GPT-5.1) |
| 3 | Gemini 3 Flash (High) | 98.8 | #159 (Gemini 3 Flash) |
| 4 | Claude Sonnet 4.5 (Thinking) | 98.7 | #244 (Claude Sonnet 4.5) |
| 5 | DeepSeek V3.2 (Thinking) | 98.7 | #403 (DeepSeek V3.2) |
| 6 | Grok 4.1 Fast (Reasoning) | 98.3 | #426 (Grok 4.1 Fast) |
| 7 | Qwen 3 235B A22B 2507 Instruct | 95.3 | #619 |
| 8 | nemotron-3-nano-30B-a3B (Reasoning) | 94.2 | #1891 (nemotron-3-nano-30B-a3B) |
| 9 | DeepSeek V3.2 (Non-reasoning) | 93.4 | #403 (DeepSeek V3.2) |
| 10 | Qwen 3 14B (Reasoning) | 90.7 | #1235 (Qwen 3 14B) |
| 11 | QwQ-32B | 89.9 | #909 |
| 12 | Llama 4 Maverick Instruct | 78.8 | #1125 |
| 13 | MiMo-V2-Flash (Non-reasoning) | 75 | #674 (MiMo-V2-Flash) |
| 14 | Ministral-3-14B-Instruct-2512 | 72 | #1595 |
| 15 | Qwen 3 14B (Non-reasoning) | 70.4 | #1235 (Qwen 3 14B) |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=nlco-set-s-feasibility-rate · How It Works · Data refreshed daily, snapshot 2026-11-12.