Kaleidoscope - Multimodal: leaderboard

Each model answers the 11,459 Kaleidoscope exam questions that need an image, in their original language across 18 languages; the score is accuracy averaged over languages with unparseable answers counted wrong. The overall Kaleidoscope board also includes the text-only questions.

Metric: Accuracy (%). Source: arxiv.org. Saturation forecast: Estimated already saturated. 8 models tracked. Updated 17 Nov 2026.

Top models

#ModelScoreOverall rank
1Qwen 2.5 VL 72B Instruct48.4#849
2Qwen 2.5 VL 32B Instruct44.9#1074
3Qwen 2.5 VL 7B Instruct36.85#1893
4Aya Vision 32B35.74#2271
5Qwen 2.5 VL 3B Instruct33.67#2648
6aya-vision-8B32.35#4874
7Molmo 7B-D31.43#5984
8Pangea-7B27.15#4993

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=kaleidoscope-multimodal · How It Works · Data refreshed daily, snapshot 2026-11-17.