PrepBench - Interactive Disambiguation: leaderboard

F1 (0-100) of the clarification questions LLM agents ask against the annotated ambiguities of each request on PrepBench (2026), a benchmark of natural-language data-preparation requests over imperfect real tables. Higher is better.

Metric: Disambiguation F1 (0-100) of the agent's clarification questions against the annotated ambiguities of each original request, PrepBench natural-language data-preparation tasks, LLM agent at temperature 0.7 with Clarify (budgeted, answered by a DeepSeek-V3.2 user simulator), Profile, Code and Translate actions; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 10 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1GPT-5.1 Codex51.4#295
2Claude Sonnet 4.551.3#261
3Gemini 3 Flash51#175
4GLM-4.748.5#380
5Kimi K2 (Thinking)45.9
6DeepSeek V3.245.7#419
7GPT-4o44.6#751
8Devstral 242.3#1010
9Qwen 3 235B A22B41.3#628
10Grok Code Fast 134.5#703

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=prepbench-interactive-disambiguation · How It Works · Data refreshed daily, snapshot 2026-11-21.