SAGE (Service Agent): leaderboard
SAGE (2026) tests LLMs as customer-service agents that must follow standard operating procedures, formalized as dialogue graphs, while handling adversarial simulated users across six industrial scenarios. The overall score weights logical compliance 0.8 and conversational quality 0.2 on a 0-100 scale, higher is better. It is unrelated to the SAGE robustness benchmark.
Metric: Overall Assessment Score (0-100), 0.8 x logical compliance + 0.2 conversational quality, averaged over the six scenarios, over SAGE customer-service dialogues: standard operating procedures formalized as dialogue graphs, an adversarial simulated user with varied intents and personas, turns 1, 5, 10, 15 and the final turn scored; three unnamed judge LLMs (majority vote and mean) plus a deterministic rule engine produce the reference; closed models through their APIs, open models served with vLLM; 0-100 scale; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 27 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.5 | 72.62 | #160 |
| 2 | Claude Sonnet 4.5 | 71.56 | #261 |
| 3 | Gemini 3 Pro (Preview) | 71.4 | #119 |
| 4 | DeepSeek V3.2 | 71.29 | #419 |
| 5 | Kimi K2.5 | 68.71 | #266 |
| 6 | DeepSeek R1 | 68.39 | #482 |
| 7 | MiniMax-M2.1 | 68.33 | #575 |
| 8 | Gemini 3 Flash (Preview) | 68.28 | #152 |
| 9 | GLM-4.7 | 67.25 | #380 |
| 10 | Gemini 2.5 Pro | 66.9 | #276 |
| 11 | GPT-4.1 | 66.79 | #505 |
| 12 | DeepSeek V3 | 66.54 | #729 |
| 13 | Qwen Max (SAGE checkpoint unspecified) | 66.54 | |
| 14 | Seed 1.8 | 66.14 | #252 |
| 15 | Qwen 2.5 32B Instruct | 65.34 | #1247 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=sage-service-agent · How It Works · Data refreshed daily, snapshot 2026-11-21.