JFinBench - No Tools - CPA: leaderboard
321examples, 25sections, 6examfamilies; mean3runs. OneAPIcall/question, 131072outputtokens(65536forGemini), nativeeffort/thinking retained. MCQ extraction; GPT5.1 judging freeanswers undernativepartialcreditrubrics; exammean overexamples, overallmeanover6exams. Failure attempts0 pernativepolicy. Public321taskset reconstructablefromofficialsourcePDFrecipes; HFaloneomits57examples/55answers. Metric: Score (%). Source: https://arxiv.org/html/2611.18130v1
Metric: Score (%). Source: arxiv.org. Saturation forecast: Around February 2027. 12 models tracked. Updated 17 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Claude Fable 5 | Max | 97.1 | #28 |
| 2 | GPT-6 Astra | Max | 96.8 | #8 |
| 3 | Gemini 3.8 Flash | High | 92.5 | #39 |
| 4 | Kimi K3.1 Pro | Max | 84.2 | #32 |
| 5 | Kimi K3 | Max | 78.9 | #59 |
| 6 | DeepSeek V4.1 Flash (1017) | Max | 73.1 | #44 |
| 7 | Qwen 3.8 Flash | Thinking | 65.9 | #128 |
| 8 | MiMo-V2.6-Pro | Thinking | 63.4 | #86 |
| 9 | GLM-5.3 Flash | Max | 56.6 | #136 |
| 10 | Mistral Large 4 | Thinking | 54.8 | #143 |
| 11 | Kimi K2.6 | Thinking | 54.5 | #191 |
| 12 | MiMo-V2.6-Flash | Thinking | 50.5 | #176 |
No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.
Interactive version: theaggregate.ai/benchmark?slug=jfinbench-no-tools-cpa · How It Works · Data refreshed daily, snapshot 2026-11-17.