MetaSyn (One-Pass RAG, BM25) - Inclusion F1: leaderboard
MetaSyn (Xie et al., 2026) asks a model to reproduce published Nature Portfolio meta-analyses: from a review's question, PI/ECO, dates and eligibility criteria it picks the included studies from a PubMed corpus and writes a synthesis report, over 86 held-out reviews. F1 in percent between the studies a report includes and the review's own included-study list, macro-averaged, when one model call reads a fixed top-200 candidate pool from BM25; higher is better.
Metric: Included-study F1 (%; articles the report includes against the source review's included-study list, macro-averaged over 86 test meta-analyses; one LLM call over a fixed top-200 BM25 pool). Source: arxiv.org. Saturation forecast: Around February 2027. 3 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.4 | 51.4 | #121 |
| 2 | DeepSeek V4 Pro | 50.6 | #181 |
| 3 | GLM-5.1 | 48.6 | #175 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=metasyn-one-pass-rag-bm25-inclusion-f1 · How It Works · Data refreshed daily, snapshot 2026-11-12.