FinMCP-Bench - Multi-Turn: leaderboard
Metric: Tool F1 (%): the harmonic mean of tool precision and tool recall between the set of MCP tools the model calls and the reference set (order and grouping ignored), with the model acting as the agent of a real fund-advisory app over its 65 financial MCP tools and each turn answered after the gold conversation history, on the 219 multi-turn samples (role-played multi-turn dialogues, expert-reviewed, each turn possibly calling tools); higher is better. Source: arxiv.org. Saturation forecast: Around 2030. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3 4B 2507 (Thinking) | 47.65 | #525 (Qwen 3 4B 2507) |
| 2 | Qwen 3 30B A3B 2507 (Thinking) | 46.4 | #366 (Qwen 3 30B A3B 2507) |
| 3 | Qwen 3 235B A22B 2507 (Thinking) | 41.56 | #253 (Qwen 3 235B A22B 2507) |
| 4 | DeepSeek R1 | 9.27 | #245 |
| 5 | GPT-OSS-20B | 7.38 | #499 |
No result here: #3 Claude Opus 5.5, #5 GPT-6 Astra, #8 Claude Fable 5.1.
Interactive version: theaggregate.ai/benchmark?slug=finmcp-bench-multi-turn · How It Works · Data refreshed daily, snapshot 2026-10-11.