CP-SynC-XL - Python + OR-Tools (Heuristic Prompt): leaderboard

Metric: Correct instances (%; share of the 4,577 CP-SynC-XL instances of 100 CSPLib and PyCSP3 combinatorial problems whose returned solution passes the reference verifier (feasibility, COP optimality) within 256 s; constraint modelling: the LLM writes a Python model for the OR-Tools CP-SAT solver, heuristic prompt that also asks the LLM to optimize search efficiency; final artefact after smoke and scaling-check refinement; greedy decoding). Source: arxiv.org. Saturation forecast: Around December 2026. 3 models tracked.

Top models

Interactive version: theaggregate.ai/benchmark?slug=cp-sync-xl-python-plus-or-tools-heuristic-prompt · How It Works · Data refreshed daily, snapshot 2026-09-26.