FDE-Gym L0 original 20: leaderboard
FDE-Gym primary paper, 13 configurations in one minimal five-tool agent framework; scripted stakeholders and held-out production data. Native business uplift minus stakeholder-contact cost; negative scores are valid, not percentages. Reference solution 1 is not a theoretical ceiling. No effort inferred where not reported; Haiku extended-thinking run stays separate. L0 original 20 task cohort; same three scenarios have identical L2/L3 instructions. Metric: Mean delivery score. Higher is better. Source: https://arxiv.org/html/2610.22350v1
Metric: Mean delivery score. Source: arxiv.org. Saturation forecast: Around December 2027. 13 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 5.5 | 0.82 | #3 |
| 2 | GPT-6.1 Sol | 0.76 | #8 |
| 3 | Claude Sonnet 5.5 | 0.74 | #10 |
| 4 | GPT-6 Astra | 0.73 | #5 |
| 5 | GLM-5.3 Flash | 0.68 | #62 |
| 6 | DeepSeek V4.1 Flash | 0.61 | #35 |
| 7 | Gemini 3.8 Flash | 0.58 | #21 |
| 8 | GPT-5.5 | 0.51 | #30 |
| 9 | Qwen 3.8 Flash | 0.36 | #47 |
| 10 | Claude Haiku 4.5 (Thinking) | 0.14 | #277 (Claude Haiku 4.5) |
| 11 | GPT-6 Luna | 0.12 | #52 |
| 12 | Claude Haiku 4.5 | 0.11 | #277 |
| 13 | Mistral Small 4 | -0.07 | #512 |
No result here: #7 Gemini 4 Argon, #9 Claude Fable 5.1.
Interactive version: theaggregate.ai/benchmark?slug=fde-gym-l0-original-20 · How It Works · Data refreshed daily, snapshot 2026-10-17.