One Word, Different Action - Decision Invariance: leaderboard
Metric: Decision invariance (%; exact match of the executable decision (object, object set or ordered action sequence) against deterministic ground truth over 85 physical decision anchors collected with a real robot; share of task-preserving instruction pairs where both decisions are correct and identical; temperature 0, at most 512 new tokens, no chain of thought requested; an unmappable output counts as wrong). Source: arxiv.org. Saturation forecast: Estimated already saturated. 4 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | GLM-5 FP8 | 96.1 |
| 2 | OLMo-2-1124-7B-Instruct | 81.4 |
| 3 | Mistral 7B Instruct (v0.3) | 75.5 |
| 4 | Qwen 2.5 7B Instruct | 65.7 |
Interactive version: theaggregate.ai/benchmark?slug=one-word-different-action-decision-invariance · How It Works · Data refreshed daily, snapshot 2026-09-26.