Chat2Workflow: leaderboard
Chat2Workflow (2026) tests whether language models build executable Dify workflows from multi-turn natural-language requests drawn from real business workflows, run against test cases. Resolve rate in percent over six domains, the share of test cases whose workflow executes and meets every requirement; higher is better.
Metric: Resolve rate (%) on Chat2Workflow (79 instructions in 27 multi-turn tasks, three test cases each, pooled over six domains): share of test cases where the workflow the model generates for the current round of a multi-turn request, deployed on Dify 1.9.2, executes and its output satisfies every requirement of the instruction, judged by GPT-5.1; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 15 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3 Pro (Preview) | 60.2 | #119 |
| 2 | Claude Sonnet 4.5 | 47.96 | #261 |
| 3 | GPT-5.2 | 47.4 | #185 |
| 4 | GLM-4.7 | 46.55 | #380 |
| 5 | Kimi K2 (Thinking) | 36 | |
| 6 | GLM-4.6 | 35.02 | #498 |
| 7 | GPT-5.1 | 34.46 | #240 |
| 8 | DeepSeek V3.1 | 32.07 | #511 |
| 9 | DeepSeek V3.2 | 27.43 | #419 |
| 10 | Kimi K2 | 24.61 | #443 |
| 11 | Qwen 3 235B A22B | 24.47 | #628 |
| 12 | Qwen 3 32B | 19.13 | #972 |
| 13 | Qwen 3 Coder 480B A35B Instruct | 19.13 | #652 |
| 14 | Qwen 3 14B | 12.38 | #1269 |
| 15 | Qwen 3 8B | 6.61 | #1873 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Example tasks
Please create a workflow dedicated to the deep reading of academic papers. The user will provide the paper file (variable paper) and the target language (variable language) as inputs. The task is to extract key information from the given paper—including background, objectives, methods, findings, and implications—in the specified target language. The output must not exceed 800 words and should be…
Interactive version: theaggregate.ai/benchmark?slug=chat2workflow · How It Works · Data refreshed daily, snapshot 2026-11-21.