Kaleidoscope: leaderboard

Each model answers all 20,911 four-option exam questions collected in their original language across 18 languages and 14 subjects, 55% of them with an image; the score is accuracy averaged over languages, with unparseable answers counted wrong. Unlike multilingual sets built by translating English benchmarks, no item is translated.

Metric: Accuracy (%). Source: arxiv.org. Saturation forecast: Estimated already saturated. 8 models tracked. Updated 17 Nov 2026.

Top models

#ModelScoreOverall rank
1Qwen 2.5 VL 72B Instruct52.94#849
2Qwen 2.5 VL 32B Instruct48.21#1074
3Qwen 2.5 VL 7B Instruct39.56#1893
4Aya Vision 32B39.27#2271
5Qwen 2.5 VL 3B Instruct35.56#2648
6aya-vision-8B35.09#4874
7Molmo 7B-D32.87#5984
8Pangea-7B31.31#4993

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=kaleidoscope · How It Works · Data refreshed daily, snapshot 2026-11-17.