Google Carbon Report - SRE-Bench - Pass@4: leaderboard
Google DeepMind Carbon safety report, Figure 7.5. 262 binary instances across malware/network protocols/firmware/game executables/compression; full-solve rate; internal non-cyber-specialized Antigravity. Trial/turn/time limits not stated. Do not infer them from SEC-bench. Metric: Pass@4 (%). Higher is better. Source: https://storage.googleapis.com/deepmind-media/gemini/gemini_4_carbon_safety_report.pdf#page=93
Metric: Pass@4 (%). Source: storage.googleapis.com. Saturation forecast: Around February 2027. 2 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 4 Carbon | 98.9 | #2 |
| 2 | Gemini 4 Argon | 58.4 | #11 |
No result here: #1 Claude Fable 5.2, #5 Claude Opus 5.5, #6 GPT-6 Astra.
Interactive version: theaggregate.ai/benchmark?slug=google-carbon-report-sre-bench-pass-4 · How It Works · Data refreshed daily, snapshot 2026-11-12.