SABRE-Prior: leaderboard
SABRE-Prior (2026) is a VLM stress test of generated and edited images whose visual evidence contradicts world priors: unexpected objects in familiar scenes, counterfactual materials, noncanonical part counts and questions that suggest an answer the image does not show. The score is the unweighted mean of four subset accuracies in percent; higher is better.
Metric: Macro-average accuracy (%; unweighted mean of the four primary subset scores: Context and Texture strict All4 accuracy, Attribute exact counting accuracy and Language Elicitation four-option accuracy; zero-shot, greedy decoding where supported, 8-16 output tokens for API models; 100 generated or edited cases per subset that survived filtering against Gemini 3.5 Flash and human review). Source: arxiv.org. Saturation forecast: Around August 2027. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4.6 | 31.3 | #161 |
| 2 | Kimi K2.6 (Non-reasoning) | 23.3 | #178 (Kimi K2.6) |
| 3 | Qwen 3.5 27B (Non-reasoning) | 23 | #359 (Qwen 3.5 27B) |
| 4 | GPT-5.4 (Non-reasoning) | 18 | #123 (GPT-5.4) |
| 5 | Grok 4.3 | 17.8 | #333 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=sabre-prior · How It Works · Data refreshed daily, snapshot 2026-11-13.