DMC-CF - Single Intervention (L1): leaderboard
Metric: Accuracy (%) on the L1 set: the annotated counterfactual questions (answer yes) and their graph-derived twins with a different condition substituted (answer no), yes/no counterfactual questions about 1,614 real-world causal event videos (videos given natively to Gemini and Qwen3-VL, as at most 16 frames to GPT and Claude); higher is better. Source: arxiv.org. Saturation forecast: Around January 2027. 5 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) | 87.25 |
| 2 | Qwen 3 VL 235B A22B (Thinking) | 82.31 |
| 3 | GPT-5.2 | 75.19 |
| 4 | Qwen 3 VL 235B A22B Instruct | 67.54 |
Interactive version: theaggregate.ai/benchmark?slug=dmc-cf-single-intervention-l1 · How It Works · Data refreshed daily, snapshot 2026-10-07.