TSHA: leaderboard
Metric: Final average (%): mean of the open-ended QA overall (GPT-4o judge rubric: 0.7 accuracy, 0.2 conciseness, 0.1 coherence) and the choice-question overall (mean of yes/no and four-option accuracy) on the 1,707-item TSHA test set (existing indoor datasets, new photos, internet and AIGC images, Sora videos, Hunyuan panoramas); higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 22 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude 3.7 Sonnet | 80.6 | #241 |
| 2 | InternVL3-8B | 80.1 | #606 |
| 3 | Claude Sonnet 4 | 80 | #194 |
| 4 | Qwen 2.5 VL 32B Instruct | 79.9 | #443 |
| 5 | Qwen 2.5 VL 7B Instruct | 76.5 | #643 |
| 6 | Gemini 2.5 Flash | 61.5 | #237 |
| 7 | Gemini 2.5 Pro | 61.4 | #145 |
| 8 | Gemma 3 27B (IT) | 39.8 | #509 |
| 9 | Gemma 3 12B (IT) | 34.7 | #655 |
| 10 | Mistral Small 3.1 | 33.6 | #600 |
| 11 | Gemma 3 4B (IT) | 33.4 | #971 |
No result here: #3 Claude Opus 5.5, #5 GPT-6 Astra, #8 Claude Fable 5.1.
Interactive version: theaggregate.ai/benchmark?slug=tsha · How It Works · Data refreshed daily, snapshot 2026-10-11.