PM-Bench (Hierarchical Multi-Agent): leaderboard
PM-Bench tests prospective memory in agents: over a simulated week the agent must remember intentions, notice the time or event that makes each one due, act then and ignore lures, while hidden state channels reveal cues only when queried. This board runs a coordinator with three specialist subagents whose suggested state queries are all executed. The score is the set-level F1 between the actions the agent selects and the actions actually due.
Metric: Set-F1 (%; F1 of the prospective actions the agent selects against the actions actually due, accumulated over the 80 steps and 81 scored tasks of the released simulated week, lures included; coordinator with three specialist subagents whose suggested state queries are all executed; one run per model). Source: arxiv.org. Saturation forecast: Around February 2027. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Mistral Large 3 | 58.1 | #817 |
| 2 | GPT-5.4 | 56.5 | #123 |
| 3 | GPT-5.3 Codex | 55.9 | #116 |
| 4 | Llama 3.3 70B Instruct | 50.9 | #1357 |
| 5 | Qwen 3 14B | 43.6 | #1233 |
| 6 | Qwen 3 32B | 37.5 | #960 |
| 7 | Mistral Small 3.2 | 33.6 | #1591 |
| 8 | Qwen 3 8B | 25.7 | #1830 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=pm-bench-hierarchical-multi-agent · How It Works · Data refreshed daily, snapshot 2026-11-13.