FDE-Gym L3 original 20: leaderboard
FDE-Gym primary paper, 13 configurations in one minimal five-tool agent framework; scripted stakeholders and held-out production data. Native business uplift minus stakeholder-contact cost; negative scores are valid, not percentages. Reference solution 1 is not a theoretical ceiling. No effort inferred where not reported; Haiku extended-thinking run stays separate. L3 original 20 task cohort; same three scenarios have identical L2/L3 instructions. Metric: Mean delivery score. Higher is better. Source: https://arxiv.org/html/2610.22350v1
Metric: Mean delivery score. Source: arxiv.org. Saturation forecast: Around December 2027. 13 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 5.5 | 0.55 | #3 |
| 2 | Claude Sonnet 5.5 | 0.46 | #10 |
| 3 | GPT-6.1 Sol | 0.43 | #8 |
| 4 | GPT-6 Astra | 0.33 | #5 |
| 5 | DeepSeek V4.1 Flash | 0.28 | #35 |
| 6 | GLM-5.3 Flash | 0.24 | #62 |
| 7 | Gemini 3.8 Flash | 0.21 | #21 |
| 8 | Qwen 3.8 Flash | 0.07 | #47 |
| 9 | GPT-5.5 | 0.06 | #30 |
| 10 | GPT-6 Luna | -0.12 | #52 |
| 11 | Claude Haiku 4.5 (Thinking) | -0.24 | #277 (Claude Haiku 4.5) |
| 12 | Claude Haiku 4.5 | -0.25 | #277 |
| 13 | Mistral Small 4 | -0.31 | #512 |
No result here: #7 Gemini 4 Argon, #9 Claude Fable 5.1.
Interactive version: theaggregate.ai/benchmark?slug=fde-gym-l3-original-20 · How It Works · Data refreshed daily, snapshot 2026-10-17.