DiagFlowBench - Termination Recognition: leaderboard
DiagFlowBench (2026) has a model guide an operator through an industrial diagnostic flowchart turn by turn, with the procedure graph in the prompt, over 1,676 conversations built from 50 manufacturer flowcharts. This board is termination recognition in percent: whether the model halts when the procedure reaches a terminal step; higher is better.
Metric: Termination recognition (%): share of terminal-node turns at which the model correctly halts the procedure (word-level Jaccard node matching), on 1,676 multi-turn maintenance conversations over 50 industrial diagnostic flowcharts from a consumer manufacturer, procedure graph given as JSON in the system prompt, temperature 0, via OpenRouter; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 10 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3 235B A22B (Thinking) | 98.5 | #623 (Qwen 3 235B A22B) |
| 2 | Nemotron 3 Super | 86.1 | #768 |
| 3 | Llama 3.3 70B | 83.4 | #1498 |
| 4 | GPT-OSS-120B | 74.4 | #700 |
| 5 | Llama 4 Maverick | 72.8 | #1099 |
| 6 | Gemini 2.5 Flash | 71.8 | #453 |
| 7 | Qwen 3 30B A3B (Thinking) | 70.1 | #1122 (Qwen 3 30B A3B) |
| 8 | Llama 4 Scout | 61.6 | #1776 |
| 9 | GPT-4o Mini | 38.8 | #1623 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=diagflowbench-termination-recognition · How It Works · Data refreshed daily, snapshot 2026-11-13.