LLM Stats (Ramp SWE-Bench v2 (Bash)): leaderboard
Ramp Labs' Ramp SWE-Bench v2.0 (Bash harness), as tracked by LLM Stats: 500 private backend engineering tasks built from production pull requests authored by over 200 Ramp engineers, evaluated as one-shot pass@1 (%) at high reasoning effort with web tools disabled and a 3.5-hour limit. Every model runs in the shared bash-only mini-swe-agent harness to isolate model differences; an anonymized multi-judge LLM verification pipeline checks whether any material difference favors the merged gold patch.
Metric: Score (%). Source: llm-stats.com. Saturation forecast: Around February 2027. 22 models tracked. Updated 19 Nov 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-6.2 Astra | 75 | #4 |
| 2 | Gemini 4 Carbon | 74 | #5 |
| 3 | Claude Fable 5.2 | 73 | #1 |
| 4 | GPT-6.2 Sol | 72.4 | #12 |
| 5 | Claude Opus 5.5 | 71.2 | #9 |
| 6 | Gemini 4 Hydrogen | 70.8 | #16 |
| 7 | Claude Fable 5.1 | 63.8 | #20 |
| 8 | GPT-6.1 Terra | 60.6 | #25 |
| 9 | Muse Wonder | 57.8 | #34 |
| 10 | Claude Sonnet 5.5 | 57.6 | #24 |
| 11 | Grok 4.7 | 55.4 | #51 |
| 12 | Gemini 4 Argon | 54.8 | #18 |
| 13 | GPT-6 Astra | 54.6 | #10 |
| 14 | Muse Ember | 54.4 | #52 |
| 15 | DeepSeek V4.1 Flash (1017) | 53.4 | #46 |
No result here: #3 Gemini 4 Neon.
Interactive version: theaggregate.ai/benchmark?slug=llm-stats-ramp-swe-bench-v2-bash · How It Works · Data refreshed daily, snapshot 2026-11-20.