DiagnosticIQ Verbose: leaderboard
DiagnosticIQ Verbose (2026) restates each symbolic sensor rule of the DiagnosticIQ maintenance-action questions in natural language. Macro accuracy in percent over the asset types, zero-shot at temperature 0; higher is better.
Metric: Macro accuracy (%) on DiagnosticIQ Verbose, the core questions with each symbolic rule paraphrased into natural language (mean of the per-asset accuracies), zero-shot multiple choice, temperature 0, at most 4,096 output tokens; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 17 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.6 | 71.58 | #113 |
| 2 | Claude 3.7 Sonnet | 68.22 | #470 |
| 3 | Claude Sonnet 4.6 | 64.81 | #173 |
| 4 | O1 | 64.53 | #403 |
| 5 | DeepSeek V3 | 62.02 | #729 |
| 6 | DiagnosticIQ Mistral-Large (checkpoint unspecified) | 60.45 | |
| 7 | Qwen 2.5 72B Instruct | 56.73 | #1117 |
| 8 | Mistral Medium 3 | 55.89 | #1096 |
| 9 | Gemini 1.5 Pro | 55.78 | #1063 |
| 10 | Llama 3.3 70B Instruct | 55.16 | #1385 |
| 11 | Llama 3.1 405B | 53.81 | #1082 |
| 12 | Mistral Small 3.1 | 53.78 | #1706 |
| 13 | Granite 3.3 8B Instruct | 53.69 | #3866 |
| 14 | Gemini 2.0 Flash | 49.78 | #723 |
| 15 | Phi-4 | 45.64 | #1931 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=diagnosticiq-verbose · How It Works · Data refreshed daily, snapshot 2026-11-21.