CAGE - Counterfactual (Text-Only Probe): leaderboard
Level-3 (counterfactual) questions of CAGE, the Causal Abstraction Gap Evaluation (Hoang and Hasan, 2026): vision-language models answer free-text causal questions about what would have happened had something been different on 500 COCO test images, three per image. Mean 0-10 rating of accuracy, relevance and logical consistency against reference answers by GPT-4o and Claude 3.5 Sonnet judges, blank answers scoring 0; higher is better.
Metric: Judge score (0-10), mean of GPT-4o and Claude 3.5 Sonnet ratings of accuracy, relevance and logical consistency against GPT-4o-generated reference answers; level-3 counterfactual (what would have happened otherwise) questions, 3 per image on 500 COCO test images; free-text answer without a causal chain (Text-Only Probe); blank responses score 0. Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | CogVLM2-19B | 8.11 | |
| 2 | Qwen-VL-Chat | 7.99 | #8066 |
| 3 | LLaVA-NeXT-13B | 7.94 | |
| 4 | InternVL2-76B | 7.12 | #1486 |
| 5 | mPLUG-Owl2 | 7.06 | #8342 |
| 6 | LLaVA-RLHF-13B | 6.92 | |
| 7 | MiniGPT-4-Vicuna-13B | 6.68 | |
| 8 | LLaVA-CoT | 4.68 | #3115 |
Interactive version: theaggregate.ai/benchmark?slug=cage-counterfactual-text-only-probe · How It Works · Data refreshed daily, snapshot 2026-11-12.