PURE - D1: leaderboard

PURE - D1 — PURE primary paper Tables2and5:1457Chinese e-commerce multiple-choice questions,14tasks,1281images. Zero-shot JSON answers; A2/A3 exact multiple-selection; C3 requires all four group responses correct; dimension/overall scores are equal task means. Main table uses the native one-decimal scores. Six paired models have explicit non-thinking/thinking configurations; the four other main rows retain their source-default condition because the paper says primarily non-thinking, not universally disabled. Thinking rows use Table5two-decimal scores; no duplicate Table5non-thinking observations or inferred dimension aggregates. Model-specific output budgets and exact serving revisions are unreported. Question-generation model Gemini3.1Pro and human data reviewers are not evaluated rows. Metric: Exact-match accuracy (%).

Metric: Exact-match accuracy (%). Source: huggingface.co. Saturation forecast: Around February 2027. 10 models tracked. Updated 15 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1Qwen 3.8 MaxThinking76.26#59
2Gemini 3.8 Flash75.5#36
3Kimi K373.4#56
4Claude Opus 572.7#20
5Qwen 3.5 27BThinking69.06#365
6GPT-5.6 Sol68.3#30
7DeepSeek V4.1 FlashNon-reasoning67.6#62
8Qwen 3.5 397B A17BThinking64.75#251
9Qwen 3.5 35B A3BThinking54.68#486
10Qwen 3.5 9BThinking53.96#814

No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.

Interactive version: theaggregate.ai/benchmark?slug=pure-d1 · How It Works · Data refreshed daily, snapshot 2026-11-16.