PM-Bench (Todo Ledger): leaderboard
PM-Bench tests prospective memory in agents: over a simulated week the agent must remember intentions, notice the time or event that makes each one due, act then and ignore lures, while hidden state channels reveal cues only when queried. This board gives the agent an in-context ledger of pending intentions that it rewrites every step. The score is the set-level F1 between the actions the agent selects and the actions actually due.
Metric: Set-F1 (%; F1 of the prospective actions the agent selects against the actions actually due, accumulated over the 80 steps and 81 scored tasks of the released simulated week, lures included; single agent that carries and rewrites an in-context ledger of pending intentions every step; one run per model). Source: arxiv.org. Saturation forecast: Around February 2027. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.3 Codex | 74.8 | #116 |
| 2 | GPT-5.4 | 73.5 | #123 |
| 3 | Llama 3.3 70B Instruct | 72.5 | #1357 |
| 4 | Mistral Large 3 | 66.7 | #817 |
| 5 | Mistral Small 3.2 | 63.4 | #1591 |
| 6 | Qwen 3 14B | 52.3 | #1233 |
| 7 | Qwen 3 32B | 51.7 | #960 |
| 8 | Qwen 3 8B | 47.8 | #1830 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=pm-bench-todo-ledger · How It Works · Data refreshed daily, snapshot 2026-11-13.