VS-Bench (VisualSwap): leaderboard
Visual re-examination benchmark (VS-Bench, Shi et al., USC and UC San Diego, 2026): a vision-language model reasons about an edited image, the image is then swapped for the near-identical original whose answer differs, and a self-reflection prompt ("Wait, let me check the figure again") is appended inside the same response before the model continues. The score is probe accuracy in percent, the share of 800 image pairs (200 from each of MathVista, MathVerse, MathVision and MMMU-Pro) answered correctly for the swapped-in image; higher is better.
Metric: Probe accuracy (%; mean over 800 image pairs, 200 from each of four sources: answer on the swapped image after a self-reflection prompt). Source: arxiv.org. Saturation forecast: Around February 2027. 15 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3 VL 32B Instruct | 61.8 | #582 |
| 2 | Qwen 3 VL 235B A22B Instruct | 61.3 | #521 |
| 3 | Qwen 3 VL 30B A3B Instruct | 49.6 | #870 |
| 4 | Qwen 3 VL 8B Instruct | 46.6 | #982 |
| 5 | Qwen 3 VL 8B (Thinking) | 36.6 | #887 |
| 6 | Qwen 3 VL 32B (Thinking) | 36.6 | #557 |
| 7 | Qwen 3 VL 235B A22B (Thinking) | 34.1 | #451 |
| 8 | Qwen 2.5 VL 7B Instruct | 29.4 | #1842 |
| 9 | Qwen 3 VL 30B A3B (Thinking) | 26.6 | #710 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=vs-bench-visualswap · How It Works · Data refreshed daily, snapshot 2026-11-12.