VistaQA - Hallucination Samples: leaderboard
Metric: Grove score (10-100): per sample the geometric mean of the answer score and the mask score, each floored at 0.1, averaged over samples and times 100, on the 314 hallucination samples whose queried entity is absent (the correct mask is none), on the 1,157 VistaQA samples (six task types, six visual domains, 314 hallucination samples whose queried entity is absent), zero-shot with standardized output-format instructions; the answer is judged correct or not by Qwen 2.5-14B and the evidence masks are matched to the reference masks by Hungarian IoU; higher is better. Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 14 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | SAM3 + Qwen3-VL-4B-Instruct | 60.72 |
| 2 | SAM3 + Qwen3-VL-32B-Instruct | 56.41 |
| 3 | SAM3 + Gemini 3 (VistaQA checkpoint unspecified) | 56.14 |
| 4 | SAM3 + GPT-5.4 (Thinking) | 47.89 |
| 5 | SAM3 + GPT-5.4 | 36.72 |
| 6 | SESAME-7B | 30.04 |
| 7 | TreeVGR-7B | 22.66 |
| 8 | R-Sa2VA-Qwen3VL-4B-RL | 20.79 |
| 9 | LISA-7B | 19.39 |
| 10 | UGround-7B | 14.03 |
Interactive version: theaggregate.ai/benchmark?slug=vistaqa-hallucination-samples · How It Works · Data refreshed daily, snapshot 2026-10-07.