SecIT Bench (Pydantic AI): leaderboard
Cribl's benchmark for IT and security operations agents, run under a plain Pydantic AI harness. Thirty telemetry scenarios across intrusion and breach, covert egress, service errors and performance degradation are each rolled out three times; an incomplete answer scores zero, and the reported figure is the mean scenario accuracy.
Metric: Accuracy (%). Source: cribl.io. Saturation forecast: Not forecast. 27 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | DeepSeek V4.1 Flash | 82.9 | #36 |
| 2 | GLM-5.3 | 81.44 | #39 |
| 3 | GPT-6 Astra | 81.01 | #6 |
| 4 | Grok 4.6 | 80.65 | #26 |
| 5 | GPT-6.1 Sol | 79.91 | #7 |
| 6 | Claude Opus 5 | 79.15 | #12 |
| 7 | Qwen 3.8 Max | 78.52 | #35 |
| 8 | Muse Spark 1.3 | 77.53 | #24 |
| 9 | Kimi K3 | 76.99 | #30 |
| 10 | Mistral Large 4 | 76.38 | #94 |
| 11 | MiMo-V2.6-Flash | 75.98 | #70 |
| 12 | GLM-5.3 Flash | 75.48 | #65 |
| 13 | GPT-5.6 Sol | 75.4 | #19 |
| 14 | Claude Sonnet 5.5 | 75.13 | #11 |
| 15 | Grok 4.5 | 74.88 | #48 |
No result here: #3 Claude Opus 5.5, #8 Gemini 4 Argon, #9 Claude Fable 5.1.
Interactive version: theaggregate.ai/benchmark?slug=secit-bench-pydantic-ai · How It Works · Data refreshed daily, snapshot 2026-10-20.