LLM Stats (Ramp SWE-Bench v2): leaderboard

Ramp Labs' Ramp SWE-Bench v2.1 (Native harness), as tracked by LLM Stats: the same 500 private backend engineering tasks from Ramp production pull requests, evaluated as one-shot pass@1 (%) at high reasoning effort with web tools disabled and a 3.5-hour limit. Each model runs in its lab's native agent harness (Codex, Claude Code, OpenCode, Muse Code, DeepSeek Harness or Grok Build); an anonymized multi-judge LLM verification pipeline checks whether any material difference favors the merged gold patch.

Metric: Score (%). Source: llm-stats.com. Saturation forecast: Around April 2027. 16 models tracked. Updated 19 Nov 2026.

Top models

#ModelScoreOverall rank
1GPT-6.2 Astra76.4#4
2Gemini 4 Carbon75.4#5
3Claude Fable 5.273.4#1
4GPT-6.2 Sol73.4#12
5Gemini 4 Hydrogen71.6#16
6GPT-6.1 Terra70.4#25
7Claude Opus 5.567.8#9
8Grok 4.764.4#51
9GPT-6 Astra63.4#10
10Claude Sonnet 5.563.2#24
11DeepSeek V4.1 Flash (1017)60.8#46
12GPT-6.1 Sol59.6#15
13Muse Wonder58.6#34
14Claude Haiku 5.557.4#75
15GPT-6 Luna52.4#106

No result here: #3 Gemini 4 Neon.

Interactive version: theaggregate.ai/benchmark?slug=llm-stats-ramp-swe-bench-v2 · How It Works · Data refreshed daily, snapshot 2026-11-20.