VIABLE - Flawed-Answer Preference: leaderboard
VIABLE (Zhao et al., 2026) evaluates vision-language models as judges of answers to video questions from visually impaired users, drawn from outdoor walking, indoor scene and hand-object manipulation corpora. Each item pairs the gold answer with a copy carrying one injected failure and is shown in both orders; a judgment counts only when the judge prefers the gold answer both times, so position bias costs points. Percentage of 33,000 pairs; higher is better.
Metric: Gold-pick rate (%; 33,000 gold-versus-failure-injected pairs, judge picks the gold response in both presentation orders, over the WAD, VisAssist and VIA-EgoDex corpora). Source: arxiv.org. Saturation forecast: Around February 2027. 7 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.4 | 65.4 | #121 |
| 2 | Claude Sonnet 4.6 | 50.2 | #158 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=viable-flawed-answer-preference · How It Works · Data refreshed daily, snapshot 2026-11-12.