FDE-Gym L3 resolved full 95: leaderboard
FDE-Gym primary paper, 13 configurations in one minimal five-tool agent framework; scripted stakeholders and held-out production data. Native business uplift minus stakeholder-contact cost; negative scores are valid, not percentages. Reference solution 1 is not a theoretical ceiling. No effort inferred where not reported; Haiku extended-thinking run stays separate. Full 95-task cohort; resolved means score at least 0.8 and no incident. Metric: Resolved tasks (out of 95). Higher is better. Source: https://arxiv.org/html/2610.22350v1
Metric: Resolved tasks (out of 95). Source: arxiv.org. Saturation forecast: Around 2029. 13 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 5.5 | 15 | #10 |
| 2 | Claude Opus 5.5 | 14 | #3 |
| 3 | GPT-6 Astra | 9 | #5 |
| 4 | DeepSeek V4.1 Flash | 9 | #35 |
| 5 | GLM-5.3 Flash | 8 | #62 |
| 6 | GPT-6.1 Sol | 7 | #8 |
| 7 | Gemini 3.8 Flash | 4 | #21 |
| 8 | Qwen 3.8 Flash | 4 | #47 |
| 9 | GPT-5.5 | 0 | #30 |
| 10 | Claude Haiku 4.5 | 0 | #277 |
| 11 | Mistral Small 4 | 0 | #512 |
| 12 | Claude Haiku 4.5 (Thinking) | 0 | #277 (Claude Haiku 4.5) |
| 13 | GPT-6 Luna | 0 | #52 |
No result here: #7 Gemini 4 Argon, #9 Claude Fable 5.1.
Interactive version: theaggregate.ai/benchmark?slug=fde-gym-l3-resolved-full-95 · How It Works · Data refreshed daily, snapshot 2026-10-17.