MultiCaption (Zero-Shot) - English Translations: leaderboard
MultiCaption (Frade et al., 2026) contradiction detection between two visual claims on the same image, using the English translations of the 4,920 test pairs. Zero-shot yes or no answer; accuracy in percent, mean of five runs at temperature 0.1; higher is better, and always answering one class scores about 50%.
Metric: Accuracy (%; the same 4,920 test pairs with the claims in their English translations; zero-shot yes/no contradiction judgment at temperature 0.1, mean of five runs). Source: arxiv.org. Saturation forecast: Estimated already saturated. 5 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Mistral 7B Instruct (v0.3) | 64.4 | #6698 |
| 2 | Qwen 2.5 7B Instruct | 58.9 | #2583 |
| 3 | Llama 3.1 8B Instruct | 53.6 | #4277 |
| 4 | gemma-7B | 49.1 | #7244 |
| 5 | Phi-4 Mini Instruct | 48.9 | #6452 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=multicaption-zero-shot-english-translations · How It Works · Data refreshed daily, snapshot 2026-11-12.