BioTool: leaderboard
Biomedical tool calling (BioTool; Gao et al., UC San Diego, 2026): given a biomedical question and the tool schemas, the model writes the NCBI, UniProt or Ensembl API call that answers it. The call is executed and its returned observation compared with the reference observation by MedCPT embedding similarity (a failed call scores 0); mean in percent over 1,408 test queries, higher is better. Proprietary and base open models are prompted in context; the BioTool-finetuned rows are the authors' models trained on the other 80% of the data.
Metric: BioTool score (%; mean MedCPT embedding similarity between the observation returned by the model's API call and the reference observation, 0 for a failed call; 1,408 test queries over NCBI, UniProt and Ensembl). Source: arxiv.org. Saturation forecast: Around April 2027. 12 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4.5 | 81.4 | #244 |
| 2 | Gemini 3 Pro | 79.2 | #135 |
| 3 | Qwen 2.5 7B Instruct | 64 | #2583 |
| 4 | Qwen 3 4B 2507 Instruct | 63.1 | #2174 |
| 5 | GPT-5.1 Codex | 56.3 | #279 |
| 6 | GPT-5.1 | 55.4 | #225 |
| 7 | Llama 3.1 8B Instruct | 54.8 | #4277 |
| 8 | Qwen 3 8B | 54.1 | #1838 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=biotool · How It Works · Data refreshed daily, snapshot 2026-11-12.