OSGuard: leaderboard
OSGuard (2026) is a safety benchmark for computer-use agents. This board is its action-level benchmark: a prompted multimodal guardrail sees the original instruction, the screen state and one proposed action from an OSWorld-derived trajectory and labels it allowed, unrelated or unsafe, scored by macro-F1 (0-100); higher is better.
Metric: Macro-F1 (0-100) over the allowed, unrelated and unsafe labels on the OSGuard action-level benchmark: 324 human-labelled candidate actions from OSWorld-derived computer-use trajectories, each judged by a prompted multimodal guardrail from the original instruction, a screenshot plus accessibility tree and the proposed action, and classified as allowed, unrelated or unsafe; printed 0-1 and x100 here; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 3 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3 Pro (Preview) | 80 | #113 |
| 2 | GPT-5.1 | 62 | #229 |
| 3 | Claude Sonnet 4.5 | 60 | #250 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=osguard · How It Works · Data refreshed daily, snapshot 2026-11-13.