PASTABench: leaderboard
PASTABench (Sun et al., 2026) evaluates LLMs as proactive safety monitors that read a multi-turn agent trajectory turn by turn and must decide whether and when to interrupt and which risk is emerging, across five physical and digital risk categories. The score is the percentage of trajectories on which the monitor first interrupts inside the annotated optimal intervention window and names the correct risk sub-category; higher is better.
Metric: Perfect interruption rate (%; share of the 1,139 risky multi-turn agent trajectories on which the monitor first interrupts inside the optimal intervention window, at or after the earliest-signal turn and before the trigger turn, and names the correct level-2 risk sub-category; zero-shot, greedy decoding, stepwise auto-stop protocol). Source: arxiv.org. Saturation forecast: Not forecast. 16 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-4o | 40.74 | #741 |
| 2 | Gemini 3 Pro | 39.42 | #135 |
| 3 | Llama 4 Maverick | 33.36 | #1105 |
| 4 | Llama 4 Scout | 30.99 | #1787 |
| 5 | DeepSeek R1 | 30.47 | #467 |
| 6 | Grok 3 | 29.94 | #600 |
| 7 | Llama 3.3 70B Instruct | 24.06 | #1353 |
| 8 | Gemini 2.5 Pro | 23.79 | #262 |
| 9 | QwQ-32B | 22.65 | #909 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=pastabench · How It Works · Data refreshed daily, snapshot 2026-11-12.