EDCT-Bench (Binary): leaderboard
EDCT-Bench (2026) measures vision-language model explanation faithfulness over 300 image-question pairs from OK-VQA, DriveLM and 3DSRBench, by extracting the visual concepts a model cites, applying verified minimal edits to them, and testing whether the answer and explanation stay consistent with the edited image. This binary board is the mean over cited concepts of answer consistency times explanation faithfulness, rescaled to percent, higher is better.
Metric: Counterfactual Consistency Score, binary (%; mean over cited concepts of answer-consistency times explanation-faithfulness after verified minimal image edits, k=2, Qwen3-235B-A22B extractor and judge, FLUX.2 Max editor). Source: arxiv.org. Saturation forecast: Around February 2027. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 2.5 Flash (Thinking) | 60 | #452 (Gemini 2.5 Flash) |
| 2 | GLM-4.6V (Reasoning) | 57 | #698 (GLM-4.6V) |
| 3 | Qwen 3 VL 8B Instruct | 51 | #982 |
| 4 | Pixtral-12B | 47 | #2493 |
| 5 | Gemma 3 27B | 44 | #1638 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=edct-bench-binary · How It Works · Data refreshed daily, snapshot 2026-11-12.