RiOSWorld - User-Requested Risks: leaderboard
RiOSWorld's seven user-originated categories (file I/O, OS operations, web, code, social media, office, multimedia): share of tasks where the agent moves to carry out a harmful or unverified user request. Here the request itself is the risk, unlike the environmental slice.
Metric: Risky-intent rate, mean of risk categories (%, lower is better). Source: arxiv.org. 10 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude 3.5 Sonnet | 66.36 | #746 |
| 2 | Llama 3.2 90B Vision Instruct | 70.73 | #1749 |
| 3 | Qwen 2.5 VL 72B Instruct | 74.99 | #846 |
| 4 | Qwen 2 VL 72B Instruct | 79.81 | #1403 |
| 5 | Claude 3.7 Sonnet | 82.36 | #458 |
| 6 | Gemini 2.5 Pro (Preview) | 85.84 | #253 |
| 7 | GPT-4o | 85.94 | #741 |
| 8 | GPT-4o Mini | 89.2 | #1623 |
| 9 | GPT-4.1 | 92.47 | #495 |
Interactive version: theaggregate.ai/benchmark?slug=riosworld-user-requested-risks · How It Works · Data refreshed daily, snapshot 2026-11-13.