RealWorldShop: leaderboard

LLMs as conversational shopping agents over full sessions against a real product catalog: about 1,200 shopping scenarios with an LLM-simulated customer who reveals and revises needs gradually. Scored by session success rate, the share of sessions that end in a grounded, actionable purchase decision within 30 assistant turns; higher is better (RealWorldShop, 2026).

Metric: Session success rate (%): share of simulated shopping sessions that end in a grounded, actionable purchase decision (Kimi-K2.6 user simulator, at most 30 assistant turns); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 7 models tracked. Updated 2 Oct 2026.

Top models

#ModelScoreOverall rank
1GPT-571#177
2Kimi K2.667#191
3DeepSeek V3.256#417
4Qwen 3.5 27B55#374
5Qwen 3.5 35B A3B53#490
6GPT-4o38#754
7Gemini 2.5 Flash38#467

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=realworldshop · How It Works · Data refreshed daily, snapshot 2026-11-17.