World Model Situational Probe Passing - Precision: leaderboard
World Model Situational Probe Passing - Precision: Feature-multiset structural agreement among probe-passing candidates (0-1). LLM-generated world models, arXiv 2611.14419v1: nine game cases times three requests per model. Game structural metrics use final artifacts including recorded repairs, permutation-invariant matching. Initial requests: temperature 0, max output 16384, reasoning disabled. Situational S1-S9 feature-multiset metrics are conditional on probe-passing candidates, denominators retained in pinned n column. Hosted IDs identify Llama 3.3 70B Instruct and DeepSeek V4 Flash 0731. Automated semantic checks are diagnostic and not human-validated; not ingested as semantic capability.
Metric: Feature-multiset structural agreement among probe-passing candidates (0-1). Source: arxiv.org. Saturation forecast: Around February 2027. 5 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3.8 27B (Non-reasoning) | 0.86 | #199 (Qwen 3.8 27B) |
| 2 | DeepSeek V4 Flash (0731) (Non-reasoning) | 0.82 | #136 (DeepSeek V4 Flash (0731)) |
| 3 | GPT-5.6 Sol (Non-reasoning) | 0.77 | #29 (GPT-5.6 Sol) |
| 4 | Claude Opus 5 (Non-reasoning) | 0.45 | #20 (Claude Opus 5) |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=world-model-situational-probe-passing-precision · How It Works · Data refreshed daily, snapshot 2026-11-12.