OpenBioRQ: leaderboard
Unsolved biomedical research questions for tool-using agents (OpenBioRQ, Jeong, 2026): the agent may take up to ten rounds of calls to ten biomedical APIs such as PubMed, ClinicalTrials.gov and UniProt, and a GLM-5.1 judge grades the answer against a frozen per-question checklist of facts to mention, gaps to acknowledge, evidence to cite and claims to avoid. The score is the percent of the 423-question frozen core (questions that all three open-weight roster models failed) answered with a weighted checklist score of at least 0.5 at temperature 0; higher is better.
Metric: Solve Rate (%; frozen 423-question core, checklist score at least 0.5, temperature 0, ten-tool harness). Source: arxiv.org. Saturation forecast: Around February 2027. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.5 | 59.6 | #48 |
| 2 | Claude Opus 4.7 | 37.8 | #75 |
| 3 | Gemini 3 Pro | 28.8 | #135 |
| 4 | Qwen 3.5 397B A17B | 16.8 | #242 |
| 5 | GLM-5 | 16.6 | #256 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=openbiorq · How It Works · Data refreshed daily, snapshot 2026-11-12.