o11y-bench - Pass@3: leaderboard
Grafana Labs' o11y-bench runs agents on 63 observability and SRE tasks, three attempts per task, with each model and reasoning setting listed separately. Pass@3 is the percentage of tasks passed on at least one of three attempts; higher is better.
Metric: Tasks passed on at least one of three attempts, Pass@3 (%). Source: o11ybench.ai. Saturation forecast: Around February 2027. 71 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-6 Luna (High) | 100 | #61 (GPT-6 Luna) |
| 2 | Claude Haiku 5.5 (High) | 100 | #43 (Claude Haiku 5.5) |
| 3 | Claude Haiku 5.5 (xHigh) | 100 | #43 (Claude Haiku 5.5) |
| 4 | Claude Haiku 5.5 (Medium) | 100 | #43 (Claude Haiku 5.5) |
| 5 | Claude Haiku 5.5 (Low) | 100 | #43 (Claude Haiku 5.5) |
| 6 | Claude Sonnet 5.5 (High) | 100 | #14 (Claude Sonnet 5.5) |
| 7 | Claude Opus 5.5 (High) | 98.44 | #3 (Claude Opus 5.5) |
| 8 | Claude Opus 5.5 (Medium) | 98.44 | #3 (Claude Opus 5.5) |
| 9 | Claude Opus 5.5 (Low) | 98.44 | #3 (Claude Opus 5.5) |
| 10 | GLM-5.3 (High) | 98.44 | #44 (GLM-5.3) |
| 11 | Gemini 4 Argon (Low) | 98.44 | #9 (Gemini 4 Argon) |
| 12 | Kimi K3 (Max) | 96.88 | #35 (Kimi K3) |
| 13 | GPT-6 Luna (Medium) | 96.88 | #61 (GPT-6 Luna) |
| 14 | Muse Spark Open (xHigh) | 96.88 | #42 (Muse Spark Open) |
| 15 | Claude Sonnet 5.5 (Max) | 96.88 | #14 (Claude Sonnet 5.5) |
No result here: #10 Claude Fable 5.1.
Interactive version: theaggregate.ai/benchmark?slug=o11y-bench-pass-3 · How It Works · Data refreshed daily, snapshot 2026-11-05.