FinMCP-Bench - Single-Tool: leaderboard
Metric: Tool F1 (%): the harmonic mean of tool precision and tool recall between the set of MCP tools the model calls and the reference set (order and grouping ignored), with the model acting as the agent of a real fund-advisory app over its 65 financial MCP tools and each turn answered after the gold conversation history, on the 145 single-tool samples (real user requests from the app's logs, each resolved by one tool call); higher is better. Source: arxiv.org. Saturation forecast: Around January 2027. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3 4B 2507 (Thinking) | 68.55 | #525 (Qwen 3 4B 2507) |
| 2 | Qwen 3 235B A22B 2507 (Thinking) | 66.85 | #253 (Qwen 3 235B A22B 2507) |
| 3 | DeepSeek R1 | 62.46 | #245 |
| 4 | Qwen 3 30B A3B 2507 (Thinking) | 51.78 | #366 (Qwen 3 30B A3B 2507) |
| 5 | GPT-OSS-20B | 22.57 | #499 |
No result here: #3 Claude Opus 5.5, #5 GPT-6 Astra, #8 Claude Fable 5.1.
Interactive version: theaggregate.ai/benchmark?slug=finmcp-bench-single-tool · How It Works · Data refreshed daily, snapshot 2026-10-11.