CISA Dense - SWE-bench Verified (Capability): leaderboard
CISA v1 AppendixA Table6, original dense models on SWE-bench Verified, temperature1. Native tasks/repeats are100/16 AA-LCR,500/3 SWE-bench,89/8 Terminal-Bench2.0,48/8 Terminal-BenchHard and220/2 GDPval. Terminal-Bench uses context summarization; BF16 weights use FP8 KV and NVFP4 weights use BF16 KV. GDPval compares against Kimi-K2.5-Thinking vadamsr2 with four Gemini3.1ProPreview judgments per rollout; missing judgments count as losses, all440 rollouts retained. Precision and missingness contracts are explicit. Reported metric: pass@1 / resolved (%).
Metric: pass@1 / resolved (%). Source: arxiv.org. Saturation forecast: Around February 2027. 2 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Nemotron 3 Ultra | 70 | #245 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=cisa-dense-swe-bench-verified-capability · How It Works · Data refreshed daily, snapshot 2026-11-05.