LLM Stats (Ramp SWE-Bench v2 (Bash)): leaderboard

Ramp Labs' Ramp SWE-Bench v2.0 (Bash harness), as tracked by LLM Stats: 500 private backend engineering tasks built from production pull requests authored by over 200 Ramp engineers, evaluated as one-shot pass@1 (%) at high reasoning effort with web tools disabled and a 3.5-hour limit. Every model runs in the shared bash-only mini-swe-agent harness to isolate model differences; an anonymized multi-judge LLM verification pipeline checks whether any material difference favors the merged gold patch.

Metric: Score (%). Source: llm-stats.com. Saturation forecast: Around February 2027. 22 models tracked. Updated 19 Nov 2026.

Top models

#ModelScoreOverall rank
1GPT-6.2 Astra75#4
2Gemini 4 Carbon74#5
3Claude Fable 5.273#1
4GPT-6.2 Sol72.4#12
5Claude Opus 5.571.2#9
6Gemini 4 Hydrogen70.8#16
7Claude Fable 5.163.8#20
8GPT-6.1 Terra60.6#25
9Muse Wonder57.8#34
10Claude Sonnet 5.557.6#24
11Grok 4.755.4#51
12Gemini 4 Argon54.8#18
13GPT-6 Astra54.6#10
14Muse Ember54.4#52
15DeepSeek V4.1 Flash (1017)53.4#46

No result here: #3 Gemini 4 Neon.

Interactive version: theaggregate.ai/benchmark?slug=llm-stats-ramp-swe-bench-v2-bash · How It Works · Data refreshed daily, snapshot 2026-11-20.