FDE-Gym L3 full 95: leaderboard
FDE-Gym primary paper, 13 configurations in one minimal five-tool agent framework; scripted stakeholders and held-out production data. Native business uplift minus stakeholder-contact cost; negative scores are valid, not percentages. Reference solution 1 is not a theoretical ceiling. No effort inferred where not reported; Haiku extended-thinking run stays separate. L3 full 95 task cohort; same three scenarios have identical L2/L3 instructions. Metric: Mean delivery score. Higher is better. Source: https://arxiv.org/html/2610.22350v1
Metric: Mean delivery score. Source: arxiv.org. Saturation forecast: Around December 2027. 13 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 5.5 | 0.56 | #3 |
| 2 | Claude Sonnet 5.5 | 0.49 | #10 |
| 3 | GPT-6 Astra | 0.38 | #5 |
| 4 | GPT-6.1 Sol | 0.37 | #8 |
| 5 | GLM-5.3 Flash | 0.22 | #62 |
| 6 | DeepSeek V4.1 Flash | 0.21 | #35 |
| 7 | Gemini 3.8 Flash | 0.19 | #21 |
| 8 | Qwen 3.8 Flash | 0.07 | #47 |
| 9 | GPT-5.5 | 0.03 | #30 |
| 10 | GPT-6 Luna | -0.12 | #52 |
| 11 | Mistral Small 4 | -0.32 | #512 |
| 12 | Claude Haiku 4.5 | -0.33 | #277 |
| 13 | Claude Haiku 4.5 (Thinking) | -0.35 | #277 (Claude Haiku 4.5) |
No result here: #7 Gemini 4 Argon, #9 Claude Fable 5.1.
Interactive version: theaggregate.ai/benchmark?slug=fde-gym-l3-full-95 · How It Works · Data refreshed daily, snapshot 2026-10-17.