LLM Stats (Ramp SWE-Bench v2): leaderboard
Ramp Labs' Ramp SWE-Bench v2.1 (Native harness), as tracked by LLM Stats: the same 500 private backend engineering tasks from Ramp production pull requests, evaluated as one-shot pass@1 (%) at high reasoning effort with web tools disabled and a 3.5-hour limit. Each model runs in its lab's native agent harness (Codex, Claude Code, OpenCode, Muse Code, DeepSeek Harness or Grok Build); an anonymized multi-judge LLM verification pipeline checks whether any material difference favors the merged gold patch.
Metric: Score (%). Source: llm-stats.com. Saturation forecast: Around April 2027. 16 models tracked. Updated 19 Nov 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-6.2 Astra | 76.4 | #4 |
| 2 | Gemini 4 Carbon | 75.4 | #5 |
| 3 | Claude Fable 5.2 | 73.4 | #1 |
| 4 | GPT-6.2 Sol | 73.4 | #12 |
| 5 | Gemini 4 Hydrogen | 71.6 | #16 |
| 6 | GPT-6.1 Terra | 70.4 | #25 |
| 7 | Claude Opus 5.5 | 67.8 | #9 |
| 8 | Grok 4.7 | 64.4 | #51 |
| 9 | GPT-6 Astra | 63.4 | #10 |
| 10 | Claude Sonnet 5.5 | 63.2 | #24 |
| 11 | DeepSeek V4.1 Flash (1017) | 60.8 | #46 |
| 12 | GPT-6.1 Sol | 59.6 | #15 |
| 13 | Muse Wonder | 58.6 | #34 |
| 14 | Claude Haiku 5.5 | 57.4 | #75 |
| 15 | GPT-6 Luna | 52.4 | #106 |
No result here: #3 Gemini 4 Neon.
Interactive version: theaggregate.ai/benchmark?slug=llm-stats-ramp-swe-bench-v2 · How It Works · Data refreshed daily, snapshot 2026-11-20.