MultiCaption (Zero-Shot): leaderboard
MultiCaption (Frade et al., 2026) asks whether two fact-checked claims about the same image or video contradict each other; the balanced test split holds 4,920 claim pairs in 52 languages drawn from the MultiClaim fact-checking corpus. Models answer yes or no zero-shot with the claims in their original languages; accuracy in percent, mean of five runs at temperature 0.1; higher is better, and always answering one class scores about 50%.
Metric: Accuracy (%; balanced test split, 4,920 claim pairs in 52 languages, claims in their original languages with the languages named in the prompt; zero-shot yes/no contradiction judgment at temperature 0.1, mean of five runs). Source: arxiv.org. Saturation forecast: Estimated already saturated. 5 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Mistral 7B Instruct (v0.3) | 56 | #6698 |
| 2 | Qwen 2.5 7B Instruct | 53.9 | #2583 |
| 3 | Llama 3.1 8B Instruct | 52.4 | #4277 |
| 4 | gemma-7B | 49.1 | #7244 |
| 5 | Phi-4 Mini Instruct | 48.7 | #6452 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=multicaption-zero-shot · How It Works · Data refreshed daily, snapshot 2026-11-12.