Eye-Q - Few-Shot CoT - English: leaderboard
Eye-Q (Najar et al., 2026), English visual word puzzles (300 synthetic scenes written by the authors and rendered with text-to-image models): each image encodes a hidden word or phrase through visual cues, and the model must output the exact target. The prompt adds three solved examples from the same subset, each with its key clues and candidate answers. Exact-match accuracy in percent; higher is better.
Metric: Exact-Match Accuracy (%; three solved examples from the same subset, English subset). Source: arxiv.org. Saturation forecast: Around February 2027. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 2.5 Pro | 44.11 | #262 |
| 2 | Grok 4.1 Fast (Reasoning) | 44.11 | #426 (Grok 4.1 Fast) |
| 3 | Gemini 2.5 Flash | 26.94 | #452 |
| 4 | GPT-5.2 (Non-reasoning) | 19.53 | #172 (GPT-5.2) |
| 5 | Qwen 3 VL 235B A22B Instruct | 14.81 | #521 |
| 6 | Llama 4 Scout | 9.76 | #1787 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=eye-q-few-shot-cot-english · How It Works · Data refreshed daily, snapshot 2026-11-12.