SAGE (Service Agent) - Ecommerce Refund: leaderboard

SAGE overall assessment score in the ecommerce refund customer-service scenario: ecommerce refunds (deep decision tree on product status and credit level), combining procedure compliance and conversational quality on a 0-100 scale, higher is better. It is unrelated to the SAGE robustness benchmark.

Metric: Overall Assessment Score (0-100), 0.8 x logical compliance + 0.2 conversational quality, in the Ecommerce Refund scenario (ecommerce refunds (deep decision tree on product status and credit level)), over SAGE customer-service dialogues: standard operating procedures formalized as dialogue graphs, an adversarial simulated user with varied intents and personas, turns 1, 5, 10, 15 and the final turn scored; three unnamed judge LLMs (majority vote and mean) plus a deterministic rule engine produce the reference; closed models through their APIs, open models served with vLLM; 0-100 scale; higher is better. Source: arxiv.org. Saturation forecast: Around May 2028. 27 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Claude Opus 4.572.73#160
2Qwen 2.5 32B Instruct72.07#1247
3DeepSeek V3.271.51#419
4Claude Sonnet 4.571.21#261
5Gemini 3 Pro (Preview)70.99#119
6GPT-4.170.32#505
7Seed 1.870.25#252
8DeepSeek R170.16#482
9Kimi K2.569.69#266
10DeepSeek V369.62#729
11Gemini 2.5 Pro69.46#276
12Qwen 2.5 7B Instruct69.05#2723
13Qwen 2.5 3B Instruct68.92#5392
14Llama 3.3 70B Instruct68.6#1385
15MiniMax-M2.168.33#575

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=sage-service-agent-ecommerce-refund · How It Works · Data refreshed daily, snapshot 2026-11-21.