RiOSWorld: leaderboard
Multimodal computer-use agents work in an Ubuntu virtual machine on 492 tasks with risks from the screen (phishing, pop-ups, reCAPTCHA, credential prompts) or from the user (harmful or careless requests); the score is the share of tasks where the agent shows intent to take the risky step, averaged over 13 categories. Unlike OS-Harm's three categories and judged traces, it uses rule-based risk detectors per category.
Metric: Risky-intent rate, mean of risk categories (%, lower is better). Source: arxiv.org. 10 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude 3.5 Sonnet | 75.97 | #746 |
| 2 | Llama 3.2 90B Vision Instruct | 79.04 | #1749 |
| 3 | Qwen 2.5 VL 72B Instruct | 82.17 | #846 |
| 4 | Claude 3.7 Sonnet | 83.1 | #458 |
| 5 | Qwen 2 VL 72B Instruct | 83.93 | #1403 |
| 6 | Gemini 2.5 Pro (Preview) | 86.52 | #253 |
| 7 | GPT-4o | 87.25 | #741 |
| 8 | GPT-4.1 | 88.88 | #495 |
| 9 | GPT-4o Mini | 90.91 | #1623 |
Interactive version: theaggregate.ai/benchmark?slug=riosworld · How It Works · Data refreshed daily, snapshot 2026-11-13.