JFinBench - No Tools - Tax Accountant: leaderboard
321examples, 25sections, 6examfamilies; mean3runs. OneAPIcall/question, 131072outputtokens(65536forGemini), nativeeffort/thinking retained. MCQ extraction; GPT5.1 judging freeanswers undernativepartialcreditrubrics; exammean overexamples, overallmeanover6exams. Failure attempts0 pernativepolicy. Public321taskset reconstructablefromofficialsourcePDFrecipes; HFaloneomits57examples/55answers. Metric: Score (%). Source: https://arxiv.org/html/2611.18130v1
Metric: Score (%). Source: arxiv.org. Saturation forecast: Around February 2027. 12 models tracked. Updated 17 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Claude Fable 5 | Max | 79.4 | #28 |
| 2 | GPT-6 Astra | Max | 73.7 | #8 |
| 3 | Gemini 3.8 Flash | High | 60.7 | #39 |
| 4 | Kimi K3 | Max | 48.3 | #59 |
| 5 | Kimi K3.1 Pro | Max | 48.1 | #32 |
| 6 | DeepSeek V4.1 Flash (1017) | Max | 39.9 | #44 |
| 7 | Qwen 3.8 Flash | Thinking | 28.8 | #128 |
| 8 | GLM-5.3 Flash | Low | 25.6 | #136 |
| 9 | Mistral Large 4 | Thinking | 25.5 | #143 |
| 10 | Kimi K2.6 | Thinking | 25.2 | #191 |
| 11 | MiMo-V2.6-Pro | Thinking | 23.1 | #86 |
| 12 | MiMo-V2.6-Flash | Thinking | 18.5 | #176 |
No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.
Interactive version: theaggregate.ai/benchmark?slug=jfinbench-no-tools-tax-accountant · How It Works · Data refreshed daily, snapshot 2026-11-17.