MMOOC - Image-Question Mismatch (Yes/No): leaderboard
MMOOC also tests answering under shifted context, where the question is still answerable from the image. This board covers yes or no questions where the image does not match the question it is paired with, and averages accuracy with a judge rating of correctness and evidence consistency in the answer.
Metric: Shifted in-context score (%; mean of accuracy on the answerable Image-Question Mismatch Yes/No questions and answer rationality, the mean 0-1 judge score of the answer's correctness and evidence consistency in steps of 0.25; judge scores averaged over GPT-5.6, Claude Opus 5 and DeepSeek-V4-Pro). Source: arxiv.org. 18 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3.5 27B | 79.25 | #359 |
| 2 | Qwen 3.5 122B A10B | 75.75 | #331 |
| 3 | Gemma 4 26B | 74 | #612 |
| 4 | Claude Opus 4.6 | 72.5 | #102 |
| 5 | Ministral 3 14B | 72.5 | #1845 |
| 6 | GPT-4o | 72 | #741 |
| 7 | Llama 4 Maverick | 71.75 | #1099 |
| 8 | O1 | 71 | #393 |
| 9 | Ministral 3 8B | 71 | #2023 |
| 10 | Gemma 4 31B | 67.75 | #394 |
| 11 | O3 | 63.75 | #228 |
| 12 | InternVL3-8B | 62 | #1688 |
| 13 | Gemini 3.1 Pro (Preview) | 59.5 | #94 |
Interactive version: theaggregate.ai/benchmark?slug=mmooc-image-question-mismatch-yes-no · How It Works · Data refreshed daily, snapshot 2026-11-13.