FDE-Gym L2 original 20: leaderboard
FDE-Gym primary paper, 13 configurations in one minimal five-tool agent framework; scripted stakeholders and held-out production data. Native business uplift minus stakeholder-contact cost; negative scores are valid, not percentages. Reference solution 1 is not a theoretical ceiling. No effort inferred where not reported; Haiku extended-thinking run stays separate. L2 original 20 task cohort; same three scenarios have identical L2/L3 instructions. Metric: Mean delivery score. Higher is better. Source: https://arxiv.org/html/2610.22350v1
Metric: Mean delivery score. Source: arxiv.org. Saturation forecast: Around January 2028. 13 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 5.5 | 0.54 | #3 |
| 2 | Claude Sonnet 5.5 | 0.47 | #10 |
| 3 | GLM-5.3 Flash | 0.38 | #62 |
| 4 | GPT-6 Astra | 0.34 | #5 |
| 5 | GPT-6.1 Sol | 0.28 | #8 |
| 6 | DeepSeek V4.1 Flash | 0.27 | #35 |
| 7 | Gemini 3.8 Flash | 0.15 | #21 |
| 8 | Qwen 3.8 Flash | 0.09 | #47 |
| 9 | GPT-5.5 | 0.04 | #30 |
| 10 | GPT-6 Luna | -0.07 | #52 |
| 11 | Claude Haiku 4.5 (Thinking) | -0.29 | #277 (Claude Haiku 4.5) |
| 12 | Claude Haiku 4.5 | -0.3 | #277 |
| 13 | Mistral Small 4 | -0.31 | #512 |
No result here: #7 Gemini 4 Argon, #9 Claude Fable 5.1.
Interactive version: theaggregate.ai/benchmark?slug=fde-gym-l2-original-20 · How It Works · Data refreshed daily, snapshot 2026-10-17.