JFinBench - No Tools - Tax Accountant: leaderboard

321examples, 25sections, 6examfamilies; mean3runs. OneAPIcall/question, 131072outputtokens(65536forGemini), nativeeffort/thinking retained. MCQ extraction; GPT5.1 judging freeanswers undernativepartialcreditrubrics; exammean overexamples, overallmeanover6exams. Failure attempts0 pernativepolicy. Public321taskset reconstructablefromofficialsourcePDFrecipes; HFaloneomits57examples/55answers. Metric: Score (%). Source: https://arxiv.org/html/2611.18130v1

Metric: Score (%). Source: arxiv.org. Saturation forecast: Around February 2027. 12 models tracked. Updated 17 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1Claude Fable 5Max79.4#28
2GPT-6 AstraMax73.7#8
3Gemini 3.8 FlashHigh60.7#39
4Kimi K3Max48.3#59
5Kimi K3.1 ProMax48.1#32
6DeepSeek V4.1 Flash (1017)Max39.9#44
7Qwen 3.8 FlashThinking28.8#128
8GLM-5.3 FlashLow25.6#136
9Mistral Large 4Thinking25.5#143
10Kimi K2.6Thinking25.2#191
11MiMo-V2.6-ProThinking23.1#86
12MiMo-V2.6-FlashThinking18.5#176

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=jfinbench-no-tools-tax-accountant · How It Works · Data refreshed daily, snapshot 2026-11-17.