DMC-CF - Single Intervention (L1): leaderboard
DMC-CF (2026) single-intervention counterfactuals: annotated questions about whether an outcome in a real-world causal video still holds under one hypothetical change, paired with variants where a different condition is changed. Yes/no accuracy in percent; higher is better.
Metric: Accuracy (%) on the L1 set: the annotated counterfactual questions (answer yes) and their graph-derived twins with a different condition substituted (answer no), yes/no counterfactual questions about 1,614 real-world causal event videos (videos given natively to Gemini and Qwen3-VL, as at most 16 frames to GPT and Claude); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 5 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) | 87.25 | #99 |
| 2 | Qwen 3 VL 235B A22B (Thinking) | 82.31 | #459 |
| 3 | DMC-CF Claude-4 (checkpoint unspecified) | 78.82 | |
| 4 | GPT-5.2 | 75.19 | #185 |
| 5 | Qwen 3 VL 235B A22B Instruct | 67.54 | #532 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=dmc-cf-single-intervention-l1 · How It Works · Data refreshed daily, snapshot 2026-11-21.