SAGE (Service Agent): leaderboard

SAGE (2026) tests LLMs as customer-service agents that must follow standard operating procedures, formalized as dialogue graphs, while handling adversarial simulated users across six industrial scenarios. The overall score weights logical compliance 0.8 and conversational quality 0.2 on a 0-100 scale, higher is better. It is unrelated to the SAGE robustness benchmark.

Metric: Overall Assessment Score (0-100), 0.8 x logical compliance + 0.2 conversational quality, averaged over the six scenarios, over SAGE customer-service dialogues: standard operating procedures formalized as dialogue graphs, an adversarial simulated user with varied intents and personas, turns 1, 5, 10, 15 and the final turn scored; three unnamed judge LLMs (majority vote and mean) plus a deterministic rule engine produce the reference; closed models through their APIs, open models served with vLLM; 0-100 scale; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 27 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Claude Opus 4.572.62#160
2Claude Sonnet 4.571.56#261
3Gemini 3 Pro (Preview)71.4#119
4DeepSeek V3.271.29#419
5Kimi K2.568.71#266
6DeepSeek R168.39#482
7MiniMax-M2.168.33#575
8Gemini 3 Flash (Preview)68.28#152
9GLM-4.767.25#380
10Gemini 2.5 Pro66.9#276
11GPT-4.166.79#505
12DeepSeek V366.54#729
13Qwen Max (SAGE checkpoint unspecified)66.54
14Seed 1.866.14#252
15Qwen 2.5 32B Instruct65.34#1247

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=sage-service-agent · How It Works · Data refreshed daily, snapshot 2026-11-21.