SWE-bench Lite (RAG Baseline): leaderboard

Each model reads a GitHub issue plus files retrieved by BM25 and writes one patch in a single generation, with no tool use; the score is the percentage of the 300 Lite issues whose patch passes the repository tests. These are the SWE-bench authors' non-agent retrieval baselines (2023 paper runs, two models added in April 2024), unlike SWE-bench Verified on our site, which uses an interactive shell agent.

Metric: Resolved (%). Source: www.swebench.com. Saturation forecast: Around 2028. 6 models tracked. Updated 17 Nov 2026.

Top models

#ModelScoreOverall rank
1Claude 3 Opus (20240229)4.33#1158
2Claude 23#2475
3GPT-4 Preview (1106)2.67#980
4SWE-Llama-7B1.33
5SWE-Llama-13B1
6GPT-3.5 Turbo 16K (0613)0.33#3432

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=swe-bench-lite-rag-baseline · How It Works · Data refreshed daily, snapshot 2026-11-17.