SGR-Bench - Row-F1: leaderboard

Row-level F1 in percent for search agents on 100 state-gated retrieval tasks, where answer-bearing records on specialized data sites appear only after the right filters and selections are set (SGR-Bench, 2026), over both formulations: a retrieved row counts only when every field is correct. Higher is better.

Metric: Row-level F1 (%) over all tasks: a row counts only when every field of the aligned row is correct, 100 state-gated retrieval tasks on specialized data sites (scholarly, official statistics, life science, environmental, regulatory); eight models in production CLI agents (GPT-5.5 in Codex CLI, the others in Claude Code) with the same Serper search, fetch and PDF tools, plus three commercial search agents; outputs parsed into rows and aligned by key fields; higher is better. Source: arxiv.org. Saturation forecast: Around April 2027. 11 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-5.543.37#56
2Deep Research41.5
3Claude Opus 4.7Medium38.51#86
4Kimi K2.5Thinking33.74#272
5GLM-5.133.32#200
6DeepSeek V4 Pro32.64#201
7Gemini 3.1 Pro (Preview)Medium27.11#102
8Seed 2.0 ProMedium20.45#226
9Qwen 3.6 Plus11.62#245
10Gemini Deep Research10.27
11Google Search AI Mode2.36

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=sgr-bench-row-f1 · How It Works · Data refreshed daily, snapshot 2026-11-22.