MARCA (English, Basic): leaderboard

MARCA (2026) tests LLM web search on 52 manually authored multi-entity questions asked in English, in the Basic framework (the model itself calls web_search via the Serper API and web_scrape). A GPT-4.1 judge checks each answer against a manually validated checklist; checklist accuracy (%) averaged over three runs, higher is better.

Metric: Checklist accuracy (0-1, times 100): share of a question's checklist items (expected entities and attributes) that a GPT-4.1 judge marks satisfied, averaged over MARCA's 52 English questions and three runs, Basic framework (the model itself calls web_search via the Serper API and web_scrape); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 14 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 3 Pro88.5#145
2GPT-5.283.7#185
3GPT-5 Mini83.7#364
4Gemini 3 Flash82.8#175
5GPT-4.181.5#505
6Kimi K276.9#443
7Sabia-474.3#440
8Sabiazinho-470.9#668
9Qwen 3 235B A22B66.8#628
10Gemini 2.5 Pro65.8#276
11GPT-4.1 Mini64#789
12Gemini 2.5 Flash52.8#463
13Qwen 3 30B A3B46.9#1156
14Gemini 2.5 Flash Lite26#964

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=marca-english-basic · How It Works · Data refreshed daily, snapshot 2026-11-21.