ARTVS - Task Type - Support Ticket Classification: leaderboard
ARTVS task type - support ticket classification on its fixed synthetic English business-document corpus: mechanical exact-answer grading across classification, invoice, spreadsheet, tool-choice and evidence tasks. Models retain their native precision, reasoning and serving budgets. Content failures include timeouts and context/output limits; format adherence uses the source's format-checked reply denominator.
Metric: Answer pass rate (%). Source: artvs.ai. Saturation forecast: Rough model projection: around 2026. 24 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen3-Coder 30B-A3B (Q4_K_M, Non-reasoning) (ARTVS Run: f8636eff133b0699) | 94.44 | |
| 2 | DeepSeek V4.1 Flash (Thinking) (ARTVS Run: f8636eff133b0699) | 88.89 | |
| 3 | Gemma 4 E4B (Q4_0, Non-reasoning) (ARTVS Run: f8636eff133b0699) | 88.89 | |
| 4 | Nemotron 3.5 Bolt E4B (Q8_0, Non-reasoning) (ARTVS Run: f8636eff133b0699) | 88.89 | |
| 5 | Qwen3 30B-A3B (Q4_K_M, Non-reasoning) (ARTVS Run: f8636eff133b0699) | 88.89 | |
| 6 | Claude Opus 5.5 (Medium) (ARTVS Run: f8636eff133b0699) | 83.33 | |
| 7 | Claude Fable 5.1 (Medium) (ARTVS Run: f8636eff133b0699) | 83.33 | |
| 8 | DeepSeek V4 Pro (0813) (Thinking) (ARTVS Run: f8636eff133b0699) | 83.33 | |
| 9 | Qwen3.8 27B (IQ3_XXS, Non-reasoning) (ARTVS Run: f8636eff133b0699) | 83.33 | |
| 10 | Claude Sonnet 5.5 (Medium) (ARTVS Run: f8636eff133b0699) | 77.78 |
Interactive version: theaggregate.ai/benchmark?slug=artvs-task-type-support-ticket-classification · How It Works · Data refreshed daily, snapshot 2026-11-05.