VistaQA - Hallucination Samples: leaderboard
VistaQA (2026) asks models for a free-form answer to a visual question together with segmentation masks of the supporting evidence, over 1,157 samples in six task types and six domains, including questions about absent objects. This board is the Grove score on the 314 samples whose queried entity is absent, where the right answer says so and predicts no mask; 10 to 100, higher is better.
Metric: Grove score (10-100): per sample the geometric mean of the answer score and the mask score, each floored at 0.1, averaged over samples and times 100, on the 314 hallucination samples whose queried entity is absent (the correct mask is none), on the 1,157 VistaQA samples (six task types, six visual domains, 314 hallucination samples whose queried entity is absent), zero-shot with standardized output-format instructions; the answer is judged correct or not by Qwen 2.5-14B and the evidence masks are matched to the reference masks by Hungarian IoU; higher is better. Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 13 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | SAM3 + Qwen3-VL-4B-Instruct | 60.72 | ||
| 2 | SAM3 + Qwen3-VL-32B-Instruct | 56.41 | ||
| 3 | SAM3 + Gemini 3 (VistaQA checkpoint unspecified) | 56.14 | ||
| 4 | SAM3 + GPT-5.4 | Thinking | 47.89 | |
| 5 | SESAME-7B | 30.04 | ||
| 6 | TreeVGR-7B | 22.66 | ||
| 7 | R-Sa2VA-Qwen3VL-4B-RL | 20.79 | ||
| 8 | LISA-7B | 19.39 | ||
| 9 | UGround-7B | 14.03 | ||
| 10 | UniPixel-7B | 12.75 | ||
| 11 | Sa2VA-8B | 11.58 | ||
| 12 | GLaMM-7B | 10.96 | ||
| 13 | LaSagnA-7B | 10.28 |
Interactive version: theaggregate.ai/benchmark?slug=vistaqa-hallucination-samples · How It Works · Data refreshed daily, snapshot 2026-11-22.