MCP-GRANITE - Tool Selection F1: leaderboard

Metric: Tool-selection F1 (%; per-run F1 of the tools the agent called against the scenario's gold tool set at that granularity, averaged over the 972 runs per model: 81 multi-step scenarios in nine edge domains, each at four tool-interface granularities from fine-grained primitives to one dispatch tool, three repetitions; a Google ADK agent over MCP with the model served locally through Ollama, temperature 0, at most 10 turns; printed as a fraction with its standard deviation and scaled to percent). Source: arxiv.org. Saturation forecast: Around December 2026. 9 models tracked.

Top models

Interactive version: theaggregate.ai/benchmark?slug=mcp-granite-tool-selection-f1 · How It Works · Data refreshed daily, snapshot 2026-09-29.