CARTBench Shopping - Uncertainty Preservation: leaderboard
Shopping advice under complex constraints, ambiguous needs and changing timelines;169 cases across17 retail domains. Expert-confirmed pass rate (%). CARTBench Shopping v1 Table2, mean expert-confirmed rate over three complete169-case evaluations. Fixed product evidence and evidence-linked per-case rubrics; no invented snapshot or reasoning setting. This is not CArtBench CuratorQA.
Metric: Expert-confirmed pass rate (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 10 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-6.1 Sol (Benchmark Sweep Run: source=CARTBenchShopping; native paper configuration; three169-case evaluations; effort and exact snapshots unreported) | 96.2 | |
| 2 | Qwen 3.8 Max (Benchmark Sweep Run: source=CARTBenchShopping; native paper configuration; three169-case evaluations; effort and exact snapshots unreported) | 93.5 | |
| 3 | DeepSeek V4 Pro (Benchmark Sweep Run: source=CARTBenchShopping; native paper configuration; three169-case evaluations; effort and exact snapshots unreported) | 93.1 | |
| 4 | MiniMax-M3 (Benchmark Sweep Run: source=CARTBenchShopping; native paper configuration; three169-case evaluations; effort and exact snapshots unreported) | 91.7 | |
| 5 | Claude Opus 5 (Benchmark Sweep Run: source=CARTBenchShopping; native paper configuration; three169-case evaluations; effort and exact snapshots unreported) | 91.2 | |
| 6 | MiMo-V2.6-Pro (Benchmark Sweep Run: source=CARTBenchShopping; native paper configuration; three169-case evaluations; effort and exact snapshots unreported) | 88.4 | |
| 7 | Kimi K3 (Benchmark Sweep Run: source=CARTBenchShopping; native paper configuration; three169-case evaluations; effort and exact snapshots unreported) | 87.5 | |
| 8 | Qwen 3.5 397B A17B (Benchmark Sweep Run: source=CARTBenchShopping; native paper configuration; three169-case evaluations; effort and exact snapshots unreported) | 78.6 | |
| 9 | Qwen 3.5 35B A3B (Benchmark Sweep Run: source=CARTBenchShopping; native paper configuration; three169-case evaluations; effort and exact snapshots unreported) | 67.1 | |
| 10 | Qwen 3.5 122B A10B (Benchmark Sweep Run: source=CARTBenchShopping; native paper configuration; three169-case evaluations; effort and exact snapshots unreported) | 65.7 |
Interactive version: theaggregate.ai/benchmark?slug=cartbench-shopping-uncertainty-preservation · How It Works · Data refreshed daily, snapshot 2026-11-06.