GraphInstruct (Generation, Zero-Shot CoT): leaderboard

GraphInstruct (Generation, Zero-Shot CoT) scores instruction-following graph generation on the 800 GraphInstruct (2026) instructions under zero-shot chain-of-thought prompting. The quality score, 0 to 100 and higher is better, is computed by code from validity, distribution match, reference similarity and constraint satisfaction across six constraint levels.

Metric: Quality S_total (0-100; the paper's 0-1 score times 100): level scores weighted 0.05, 0.10, 0.15, 0.20, 0.25, 0.25 for L0 to L5, each level score a weighted mix of structural validity and distribution match, reference similarity and instruction-constraint satisfaction checked by code (no LLM judge; efficiency excluded), over the 800 instructions of GraphInstruct (instruction-following graph generation in six constraint levels); zero-shot chain-of-thought prompting; five generations per instruction at temperature 0.7 with the InstructGraph code-style prefix; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 11 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Claude Sonnet 4.687.8#179
2Qwen 3.5 397B A17B86.2#270
3Qwen 3.5 122B A10B86.1#350
4Qwen 3.5 35B A3B84.6#494
5GraphInstruct DeepSeek-V3 (checkpoint unspecified)82.9
6Llama 3.3 70B Instruct82.3#1394
7GPT-4o80.8#754
8GPT-4.180.2#506
9GPT-3.5 Turbo78.6#3076
10GPT-4o Mini78.4#1680
11Llama 3.1 8B Instruct73.1#4655

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=graphinstruct-generation-zero-shot-cot · How It Works · Data refreshed daily, snapshot 2026-11-22.