Executable Games - Counterfactual Revision: leaderboard
Executable Games (Fan et al., 2026) in which the environment answers the model's second query wrongly and corrects it on the next turn, so the model has to revise what it inferred. Percent of games solved; higher is better.
Metric: Success Rate (%; the second query is answered wrongly, then corrected). Source: arxiv.org. Saturation forecast: Around February 2027. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) | 57.56 | #91 |
| 2 | DeepSeek V3.2 | 40.21 | #403 |
| 3 | GPT-5.4 | 36.69 | #121 |
| 4 | Qwen 3 Max | 35.99 | #401 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=executable-games-counterfactual-revision · How It Works · Data refreshed daily, snapshot 2026-11-12.