FDE Requirement Discovery - Moderate ambiguity: leaderboard
Enterprise text-to-SQL FDE study Table 1:10 independent simulated-customer discovery trajectories per model and ambiguity level; recovery of 17 known core requirements, judged against frozen annotations. The three ambiguity levels and their mean remain separate. Dialogue counts do not measure capability and delivery-agent training outcomes are different protocols. No unreported reasoning setting is inferred. Metric: Recovery rate (%).
Metric: Recovery rate (%). Source: arxiv.org. Saturation forecast: Around February 2027. 15 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 5.5 | 84.4 | #3 |
| 2 | Claude Opus 5 | 81.3 | #16 |
| 3 | Claude Sonnet 5.5 | 79.4 | #14 |
| 4 | GPT-6.1 Sol | 76.9 | #7 |
| 5 | Claude Fable 5 | 76.3 | #20 |
| 6 | Grok 4.7 | 63.1 | #26 |
| 7 | Gemini 2.5 Flash | 62.5 | #255 |
| 8 | GPT-5.6 Sol | 60.6 | #22 |
| 9 | Llama 3.3 70B | 55.6 | #578 |
| 10 | GPT-5.4 Mini | 46.3 | #228 |
| 11 | GLM-5.3 Flash | 46.3 | #77 |
| 12 | Grok 4.20 (Reasoning) | 45.6 | #179 (Grok 4.20) |
| 13 | Command A | 41.3 | #498 |
| 14 | GPT-OSS-120B | 39.4 | #346 |
| 15 | Gemma 4 31B | 37.5 | #212 |
No result here: #6 GPT-6 Astra, #9 Gemini 4 Argon, #10 Claude Fable 5.1.
Interactive version: theaggregate.ai/benchmark?slug=fde-requirement-discovery-moderate-ambiguity · How It Works · Data refreshed daily, snapshot 2026-11-05.