SIMMER: leaderboard
SIMMER (2026) tests LLM planners on 100 cooking tasks by executing each complete plan in a human-curated symbolic kitchen world model (77 actions, 262 objects) that flags immediate failures and latent failures, valid steps that leave hazardous or corrupted state. The board is the percent of plans with any failure under the baseline prompt; lower is better.
Metric: Plans with any failure (% of the 100 tasks); each model writes a complete cooking plan for 100 kitchen tasks with the baseline prompt (no refinement), and the plan is executed step by step in a human-curated symbolic world model (77 actions, 262 objects) whose state machine flags immediate failures (violated preconditions) and latent failures (valid steps that leave hazardous or corrupted state); lower is better. Source: arxiv.org. Saturation forecast: Around May 2027. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3 Flash (Preview) | 83 | #144 |
| 2 | GPT-5.4 (Low) | 87 | #123 (GPT-5.4) |
| 3 | Claude Opus 4.6 | 89 | #102 |
| 4 | DeepSeek V3.2 (Non-reasoning) | 99 | #406 (DeepSeek V3.2) |
| 5 | Qwen 3.5 27B (Non-reasoning) | 99 | #359 (Qwen 3.5 27B) |
| 6 | Llama 3.3 70B Instruct | 100 | #1357 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=simmer · How It Works · Data refreshed daily, snapshot 2026-11-13.