JFinBench - Tools - mini-SWE-agent 2.4.6: leaderboard

Same321examplesandGPT5.1grading, mean3runs; terminal/webtools; fixednamedharness; highestnativeeffort. Allcompleted/failedattemptsperAppendixD, notbest-ofharnessselection. PreserveDS1017checkpoint; noSPRO/reasoningdefaults inferred. Metric: Overall score (%). Source: https://arxiv.org/html/2611.18130v1

Metric: Overall score (%). Source: arxiv.org. Saturation forecast: Estimated already saturated. 12 models tracked. Updated 17 Nov 2026.

Top models

#ModelScoreOverall rank
1GPT-6 Astra96.8#8
2Claude Fable 596.3#28
3Gemini 3.8 Flash94#39
4DeepSeek V4.1 Flash (1017)91.5#44
5Kimi K390.1#59
6MiMo-V2.6-Pro86.3#86
7Qwen 3.8 Flash85.9#128
8Kimi K3.1 Pro84.9#32
9GLM-5.3 Flash77#136
10Kimi K2.674.7#191
11MiMo-V2.6-Flash72.9#176
12Mistral Large 465#143

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=jfinbench-tools-mini-swe-agent-2-4-6 · How It Works · Data refreshed daily, snapshot 2026-11-17.