CausalDS: leaderboard
Metric: CausalDS score (0 is perfect, lower is better, unbounded above; task-count-weighted sum of one minus the pass rate, the relative numeric error S_NR and the median F1 loss, on a 100-scene realistic exam of Pearl-ladder causal data-science tasks (28 associational, 51 interventional, 21 counterfactual) over SCM-generated data files, solved in the mini-swe-agent harness with at most 100 steps). Source: arxiv.org. Saturation forecast: Around October 2027. 6 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | Claude Opus 4.8 (High) | 0.28 |
| 2 | Gemini 3.1 Pro (Preview) | 0.37 |
| 3 | Qwen 3.6 35B A3B | 0.45 |
| 4 | Kimi K2.6 | 0.48 |
| 5 | GPT-5.5 (High) | 0.56 |
| 6 | Gemma 4 26B | 0.64 |
Interactive version: theaggregate.ai/benchmark?slug=causalds · How It Works · Data refreshed daily, snapshot 2026-09-29.