PURE - Perception: leaderboard
PURE - Perception — PURE primary paper Tables2and5:1457Chinese e-commerce multiple-choice questions,14tasks,1281images. Zero-shot JSON answers; A2/A3 exact multiple-selection; C3 requires all four group responses correct; dimension/overall scores are equal task means. Main table uses the native one-decimal scores. Six paired models have explicit non-thinking/thinking configurations; the four other main rows retain their source-default condition because the paper says primarily non-thinking, not universally disabled. Thinking rows use Table5two-decimal scores; no duplicate Table5non-thinking observations or inferred dimension aggregates. Model-specific output budgets and exact serving revisions are unreported. Question-generation model Gemini3.1Pro and human data reviewers are not evaluated rows. Metric: Exact-match accuracy (%).
Metric: Exact-match accuracy (%). Source: huggingface.co. Saturation forecast: Around February 2027. 10 models tracked. Updated 15 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Gemini 3.8 Flash | 84.7 | #36 | |
| 2 | Qwen 3.8 Max | Non-reasoning | 74.2 | #59 |
| 3 | Kimi K3 | 71.3 | #56 | |
| 4 | Claude Opus 5 | 69.9 | #20 | |
| 5 | GPT-5.6 Sol | 69.5 | #30 | |
| 6 | DeepSeek V4.1 Flash | Non-reasoning | 63.7 | #62 |
| 7 | Qwen 3.5 397B A17B | Non-reasoning | 54 | #251 |
| 8 | Qwen 3.5 27B | Non-reasoning | 53.1 | #365 |
| 9 | Qwen 3.5 35B A3B | Non-reasoning | 42.6 | #486 |
| 10 | Qwen 3.5 9B | Non-reasoning | 39.5 | #814 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=pure-perception · How It Works · Data refreshed daily, snapshot 2026-11-16.