CI-Work - Violation Rate: leaderboard
CI-Work (2026) tests whether enterprise LLM agents, retrieving dense workplace context through simulated email, chat, calendar and meeting tools, convey what a task needs while withholding sensitive entries that contextual-integrity norms forbid sharing; a GPT-5.2 judge marks each disclosed entry. Share of cases whose final action discloses at least one sensitive entry, averaged over the five information-flow directions, in percent; lower is better.
Metric: Violation rate (%): share of cases whose final action discloses at least one sensitive entry, mean of the five information-flow directions, on CI-Work enterprise scenarios (125 seeds, 25 per information-flow direction, each with 4 sensitive and 4 essential entries retrieved through simulated email, chat, calendar and meeting tools), ReAct agent, disclosure judged per entry by a GPT-5.2 judge (83 to 91 percent agreement with human labels); lower is better. Source: arxiv.org. Saturation forecast: Around February 2027. 9 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | CI-Work DeepSeek-R1 (checkpoint unspecified) | 15.8 | |
| 2 | GPT-4o | 21.33 | #751 |
| 3 | CI-Work DeepSeek-V3 (checkpoint unspecified) | 21.33 | |
| 4 | GPT-4.1 | 25.91 | #505 |
| 5 | GPT-5 | 27.83 | #178 |
| 6 | CI-Work Kimi-K2 (checkpoint unspecified) | 31.06 | |
| 7 | O3 | 32.2 | #237 |
| 8 | Qwen 2.5 32B (CI-Work checkpoint unspecified) | 42.57 | |
| 9 | Grok 3 (CI-Work checkpoint unspecified) | 50.87 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=ci-work-violation-rate · How It Works · Data refreshed daily, snapshot 2026-11-21.