DecisionRAG MD2D - R@1: leaderboard
DecisionRAG v1 uses Jev (typesafe/jev-1.13), dense retrieval with Qwen3-Embedding-8B, BGE (bge-reranker-v2-m3), a Qwen3 8B reranker, and GPT 5 with low reasoning. Evaluation populations are MD2D with 4201 validation turns, CQA with 271 evidence-labeled development questions, MTRAG with 188 labeled IBM Cloud turns, and NQ with 500 seeded DAPR test questions. Main MTRAG gold labels are expanded using GPT 5 to judge the pooled top 5 documents; comparisons using the original official labels remain separate. Trained judges use Qwen3 Base at 1.7B, 4B, and 8B; their results are distinct from backbone language-model scores. All source-qualified harnesses and training variants remain separate. Metric: Accuracy (%).
Metric: Accuracy (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 23 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | DecisionRAG trained 8B / general → \to MD2D 10pages | 65.8 | |
| 2 | DecisionRAG trained 8B / MD2D only 10pages | 64.3 | |
| 3 | DecisionRAG trained 8B / general → \to MD2D 3pages | 63.5 | |
| 4 | DecisionRAG trained 8B / MD2D only 3pages | 63.1 | |
| 5 | DecisionRAG study DecisionRAG (8B trained), 10 pages | 60.8 | |
| 6 | DecisionRAG study DecisionRAG (8B trained), 3 pages | 60.7 | |
| 7 | DecisionRAG study DPR + cross-encoder ( Zhang et al., 2026 ) † | 55 | |
| 8 | DecisionRAG study DPR fine-tuned ( Feng et al., 2021 ) † | 49 | |
| 9 | DecisionRAG study DecisionRAG (Jev) + dense candidates | 48.5 | |
| 10 | DecisionRAG study Dense top-20 + GPT-5 s | 47.5 |
Interactive version: theaggregate.ai/benchmark?slug=decisionrag-md2d-r-1 · How It Works · Data refreshed daily, snapshot 2026-10-28.