PhysCount - Score: leaderboard
PhysCount (arXiv 2610.40530): 4,595 human-written counting questions on 546 real photos with mirrors and glass, where one physical object can appear several times by reflection or transmission. Unlike ordinary counting sets, it asks for appearances and for the physical entities behind them; Score is the group-macro product of one minus normalized error across the four question families (higher is better).
Metric: Score (%): group-macro product of one minus NAE over question families. Source: arxiv.org. Saturation forecast: Around 2031. 40 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.7 Flash | 36.45 | #24 |
| 2 | Muse Glimmer 30B | 29.61 | #191 |
| 3 | Qwen 3.8 27B | 28.84 | #123 |
| 4 | Qwen 3.5 397B A17B | 27.95 | #154 |
| 5 | Gemma 4 31B (IT) | 26.32 | #178 |
| 6 | Qwen 3.5 27B | 26.32 | #192 |
| 7 | Qwen 3.5 122B A10B | 26.03 | #184 |
| 8 | Qwen 3.5 35B A3B | 25.7 | #268 |
| 9 | Gemma 4 26B A4B (IT) | 23.79 | #293 |
| 10 | Qwen 3.5 9B | 23.54 | #391 |
| 11 | GPT-5.6 Sol | 21.73 | #22 |
| 12 | GPT-6 Astra | 20.77 | #6 |
| 13 | GPT-5.6 Terra | 20.42 | #40 |
| 14 | Gemma 4 12B (IT) | 19.03 | #332 |
| 15 | Qwen 3.5 4B | 18.57 | #514 |
No result here: #3 Claude Opus 5.5, #7 GPT-6.1 Sol, #9 Gemini 4 Argon.
Interactive version: theaggregate.ai/benchmark?slug=physcount-score · How It Works · Data refreshed daily, snapshot 2026-11-05.