JFinBench - Tools - mini-SWE-agent 2.4.6: leaderboard
Same321examplesandGPT5.1grading, mean3runs; terminal/webtools; fixednamedharness; highestnativeeffort. Allcompleted/failedattemptsperAppendixD, notbest-ofharnessselection. PreserveDS1017checkpoint; noSPRO/reasoningdefaults inferred. Metric: Overall score (%). Source: https://arxiv.org/html/2611.18130v1
Metric: Overall score (%). Source: arxiv.org. Saturation forecast: Estimated already saturated. 12 models tracked. Updated 17 Nov 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-6 Astra | 96.8 | #8 |
| 2 | Claude Fable 5 | 96.3 | #28 |
| 3 | Gemini 3.8 Flash | 94 | #39 |
| 4 | DeepSeek V4.1 Flash (1017) | 91.5 | #44 |
| 5 | Kimi K3 | 90.1 | #59 |
| 6 | MiMo-V2.6-Pro | 86.3 | #86 |
| 7 | Qwen 3.8 Flash | 85.9 | #128 |
| 8 | Kimi K3.1 Pro | 84.9 | #32 |
| 9 | GLM-5.3 Flash | 77 | #136 |
| 10 | Kimi K2.6 | 74.7 | #191 |
| 11 | MiMo-V2.6-Flash | 72.9 | #176 |
| 12 | Mistral Large 4 | 65 | #143 |
No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.
Interactive version: theaggregate.ai/benchmark?slug=jfinbench-tools-mini-swe-agent-2-4-6 · How It Works · Data refreshed daily, snapshot 2026-11-17.