Seeing What Matters - Strict-Clean VLM - Wrong-command Evidence: leaderboard
Seeing What Matters v1, strict-clean full-candidate VLM stress test: 12 frozen RGB candidate views, exactly 3 unique ranked view identifiers, 1500 evaluator episodes per VLM; simulator postconditions hidden from selector. These results are not the sequential active-acquisition policy. Source-native model/endpoint labels retained; effort and exact served snapshot unreported.
Metric: Score (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 7 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Amazon Nova Pro (Seeing What Matters Run: strict-clean;views=12;select=3;effort=unreported) | 20.3 | |
| 2 | Qwen3-VL-8B (Seeing What Matters Run: strict-clean;views=12;select=3;effort=unreported) | 20.7 | |
| 3 | Grok 4.6 (Seeing What Matters Run: strict-clean;views=12;select=3;effort=unreported) | 27.8 | |
| 4 | Gemma 3 27B (Seeing What Matters Run: strict-clean;views=12;select=3;effort=unreported) | 28.1 | |
| 5 | Grok 4.3 (Seeing What Matters Run: strict-clean;views=12;select=3;effort=unreported) | 28.2 | |
| 6 | Llama 4 Maverick (Seeing What Matters Run: strict-clean;views=12;select=3;effort=unreported) | 32.3 | |
| 7 | Sonar Pro (Seeing What Matters Run: strict-clean;views=12;select=3;effort=unreported) | 34.4 |
Interactive version: theaggregate.ai/benchmark?slug=seeing-what-matters-strict-clean-vlm-wrong-command-evidence · How It Works · Data refreshed daily, snapshot 2026-11-12.