EMem-Bench - Dynamic Tracking: leaderboard
The Dynamic Tracking family of EMem-Bench (Liang et al., 2026): after a long history of household activity in AI2-THOR and ProcTHOR homes, the agent must act on the latest location or state of an object that moved or changed during the history. Success rate in percent over 1,052 episodes, with the full history given as text and at most eight action steps; higher is better.
Metric: Success rate (%; the 1052 Dynamic Tracking episodes, acting on the latest location or state of an object that moved or changed; full-context condition: the complete interaction history as sanitized text plus the current 500 x 500 RGB view, one primitive action per call with deterministic decoding and at most eight target-task steps in AI2-THOR and ProcTHOR homes; an episode succeeds only if the simulator reaches the target terminal state). Source: arxiv.org. Saturation forecast: Around February 2027. 17 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3 Flash | 53.9 | #159 |
| 2 | Gemini 2.5 Pro | 51.6 | #262 |
| 3 | Qwen 3.6 27B | 49.5 | #339 |
| 4 | GPT-5.4 | 46.7 | #121 |
| 5 | Qwen 3 VL 4B Instruct | 37.3 | #1281 |
| 6 | InternVL3-8B | 31.8 | #1695 |
| 7 | Qwen 3 VL 8B Instruct | 26.6 | #982 |
| 8 | GPT-5.4 Mini | 25.8 | #389 |
| 9 | Mistral Small 3.1 | 25.7 | #1699 |
| 10 | Qwen 3 VL 32B Instruct | 20.3 | #582 |
| 11 | InternVL3-38B | 17 | #905 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=emem-bench-dynamic-tracking · How It Works · Data refreshed daily, snapshot 2026-11-12.