Chat2Workflow: leaderboard

Chat2Workflow (2026) tests whether language models build executable Dify workflows from multi-turn natural-language requests drawn from real business workflows, run against test cases. Resolve rate in percent over six domains, the share of test cases whose workflow executes and meets every requirement; higher is better.

Metric: Resolve rate (%) on Chat2Workflow (79 instructions in 27 multi-turn tasks, three test cases each, pooled over six domains): share of test cases where the workflow the model generates for the current round of a multi-turn request, deployed on Dify 1.9.2, executes and its output satisfies every requirement of the instruction, judged by GPT-5.1; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 15 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 3 Pro (Preview)60.2#119
2Claude Sonnet 4.547.96#261
3GPT-5.247.4#185
4GLM-4.746.55#380
5Kimi K2 (Thinking)36
6GLM-4.635.02#498
7GPT-5.134.46#240
8DeepSeek V3.132.07#511
9DeepSeek V3.227.43#419
10Kimi K224.61#443
11Qwen 3 235B A22B24.47#628
12Qwen 3 32B19.13#972
13Qwen 3 Coder 480B A35B Instruct19.13#652
14Qwen 3 14B12.38#1269
15Qwen 3 8B6.61#1873

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Example tasks

Please create a workflow dedicated to the deep reading of academic papers. The user will provide the paper file (variable paper) and the target language (variable language) as inputs. The task is to extract key information from the given paper—including background, objectives, methods, findings, and implications—in the specified target language. The output must not exceed 800 words and should be…

Source

Interactive version: theaggregate.ai/benchmark?slug=chat2workflow · How It Works · Data refreshed daily, snapshot 2026-11-21.