MerchantBench (Hermes Agent): leaderboard
MerchantBench (Alibaba and Zhejiang University, 2026) tests long-term coherence by having an LLM agent run a drop-shipping online store for 365 simulated days, sourcing products from real catalog records, pricing listings, managing cash flow and reacting to delayed order outcomes. The agent runs in the Hermes Agent harness with its own code, planning, memory and skill tools; final net assets in thousands of RMB, mean of three runs, higher is better.
Metric: Final net assets (thousand RMB; 365-day order-level simulation of a drop-shipping store grounded in 98,843 real product records, 26 tools for sourcing, listing, pricing and cash flow; the store starts with RMB 2,000 cash and a RMB 1,000 deposit; mean of three runs; the Nous Research Hermes Agent default configuration with its code, planning, memory and skill tools). Source: arxiv.org. Saturation forecast: Around September 2027. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3.7 Max | 59.46 | #145 |
| 2 | GPT-5.6 Sol | 52.93 | #30 |
| 3 | GLM-5.2 | 42.32 | #138 |
| 4 | Claude Opus 4.8 | 35.56 | #56 |
| 5 | Qwen 3.7 Plus | 29.42 | #207 |
| 6 | DeepSeek V4 Flash | 24.69 | #258 |
| 7 | Kimi K2.6 | 23.96 | #178 |
| 8 | DeepSeek V4 Pro | 16.71 | #188 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=merchantbench-hermes-agent · How It Works · Data refreshed daily, snapshot 2026-11-13.