ChronosBench - Complex Scenes (Guided Prompt): leaderboard
ChronosBench (Long-term Task-oriented Agent, Shi et al., 2026) runs a model as a proactive task agent in simulated multi-turn dialogues: a GPT-4.1 user states a need, adds constraints and, in complex scenes, shifts intent while an external environment updates over a simulated timeline, and the agent must set reminders, notify the user when an update meets the current need and otherwise stay silent. The score is the task completion rate on 54 complex cases, the equal-weight mean of the notify and stay-silent branches (27 cases each), in percent; higher is better. On this board every model gets the basic prompt plus the paper's extra guidance on the JSON response format and on when to complete a task, with thinking disabled.
Metric: Task Completion Rate (%; equal-weight mean of positive and negative branches, 54 complex cases, basic prompt plus guidance, thinking disabled). Source: arxiv.org. Saturation forecast: Estimated already saturated. 7 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4 (Non-reasoning) | 72.22 | #397 (Claude Sonnet 4) |
| 2 | GPT-4.1 | 61.11 | #492 |
| 3 | Qwen 3 8B (Non-reasoning) | 16.67 | #1838 (Qwen 3 8B) |
| 4 | Llama 3.1 8B Instruct | 11.11 | #4277 |
| 5 | Qwen 3 32B (Non-reasoning) | 11.11 | #960 (Qwen 3 32B) |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=chronosbench-complex-scenes-guided-prompt · How It Works · Data refreshed daily, snapshot 2026-11-12.