FORTIS: leaderboard
Large language model agents increasingly operate through an intermediate skill layer that mediates between user intent and concrete task execution. This layer is widely treated as an organizational abstraction, but we argue it is also a privilege boundary that current models routinely exceed. We present FORTIS, a benchmark that evaluates over-privilege in agent skills across two stages:
Metric: Task 1 EM (self-reported). Source: benchmarklist.com. Saturation forecast: Not forecast. 10 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.7 | 54.8 | #49 |
| 2 | Qwen 3.6 Max | 51.3 | #96 |
| 3 | Claude Sonnet 4.6 | 49.5 | #90 |
| 4 | Gemini 3.1 Pro (Preview) | 47.7 | #58 |
| 5 | Gemini 3 Flash | 46.2 | #99 |
| 6 | DeepSeek V4 Flash | 46.2 | #137 |
| 7 | Kimi K2.6 | 44 | #104 |
| 8 | GPT-5.4 Mini | 43.5 | #212 |
| 9 | GPT-5.4 | 42.2 | #79 |
| 10 | GPT-5.5 | 41.2 | #31 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=fortis · How It Works · Data refreshed daily, snapshot 2026-10-20.