ConsumerSimBench - Negative Aspects: leaderboard

ConsumerSimBench Negative Aspects measures whether the comments a model writes about 1,553 real RedNote trending topics reproduce the negative aspects that real consumers expressed (ConsumerSimBench, 2026). The score is the percent of the criteria in that family that the comments cover, judged by GPT-5.2; higher is better.

Metric: Criterion coverage (%) for T4 negative aspects (the event details consumers criticised): the share of a topic's criteria that the model's generated comment set covers, each criterion judged pointwise by GPT-5.2 against its definition and examples, averaged over the 1,553 RedNote trending topics of ConsumerSimBench (23,122 criteria in all); the model writes 10 to 30 realistic comments per topic from a neutral event description, one call at temperature 0.7; higher is better. Source: arxiv.org. Saturation forecast: Around June 2028. 13 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Gemini 3.1 Pro (Preview)31.1#102
2ConsumerSimBench DeepSeek-V3 (checkpoint unspecified)30.6
3Kimi K2.530.4#272
4Grok 4.2027.2#307
5Gemini 3 Flash26.8#180
6GPT-5.225.6#190
7Qwen 3.5 397B A17B23.9#270
8Doubao Seed 1.8Thinking22.1#359
9MiMo-V2.5-Pro20.1#292
10Claude Opus 4.619.8#114
11MiniMax-M2.519.6#600
12Claude Sonnet 4.616.1#179
13GPT-4o12.1#754

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=consumersimbench-negative-aspects · How It Works · Data refreshed daily, snapshot 2026-11-22.