Visual Sanad - De-biased prompt discrimination: leaderboard
Visual Sanad locked 2,400-image set (100 English Qur'an verses x four factual states x six visual conditions, from plain text to decorative and authority-framed layouts); the VLM decides whether the displayed quote belongs to the displayed reference. Discrimination = 100 - invalid-citation false-accept rate - valid-citation false-reject rate (percentage points; a constant VALID answer scores near zero, and it can be negative). De-biased prompt: the natural prompt plus the instruction not to treat decorative design, religious-looking borders or verification headers as evidence that the citation is correct. Open models use greedy decoding (do_sample=False, seed 31); model labels as printed, API checkpoint dates and reasoning settings not disclosed. The false-accept rate alone is not published as a board because a model that rejects every citation would minimise it. Metric: Discrimination (pp). Higher is better. Source: https://arxiv.org/html/2610.22009v1
Metric: Discrimination (pp). Source: arxiv.org. Saturation forecast: Estimated already saturated. 12 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.5 | 99.67 | #31 |
| 2 | GPT-5.6 Terra | 95.11 | #41 |
| 3 | GPT-4.1 | 85.44 | #245 |
| 4 | GPT-4o | 84.5 | #340 |
| 5 | Gemma 4 12B (IT) | 2 | #381 |
| 6 | Qwen 2.5 VL 7B | 0.11 | #642 |
No result here: #4 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=visual-sanad-de-biased-prompt-discrimination · How It Works · Data refreshed daily, snapshot 2026-10-18.