SWE-bench Lite (RAG Baseline): leaderboard
Each model reads a GitHub issue plus files retrieved by BM25 and writes one patch in a single generation, with no tool use; the score is the percentage of the 300 Lite issues whose patch passes the repository tests. These are the SWE-bench authors' non-agent retrieval baselines (2023 paper runs, two models added in April 2024), unlike SWE-bench Verified on our site, which uses an interactive shell agent.
Metric: Resolved (%). Source: www.swebench.com. Saturation forecast: Around 2028. 6 models tracked. Updated 17 Nov 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude 3 Opus (20240229) | 4.33 | #1158 |
| 2 | Claude 2 | 3 | #2475 |
| 3 | GPT-4 Preview (1106) | 2.67 | #980 |
| 4 | SWE-Llama-7B | 1.33 | |
| 5 | SWE-Llama-13B | 1 | |
| 6 | GPT-3.5 Turbo 16K (0613) | 0.33 | #3432 |
No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.
Interactive version: theaggregate.ai/benchmark?slug=swe-bench-lite-rag-baseline · How It Works · Data refreshed daily, snapshot 2026-11-17.