ACARe Agentic TRQA - nDCG@10: leaderboard
ACARe v1 is a fine-tuned Qwen3-Reranker-4B, using full fine-tuning in bf16 on 16,601 groups. RetrievalBench's one-shot and agentic populations use BCP with 200 questions (seed 42, averaging gold and evidence qrels), BRIGHT with 1384 questions across 12 domains, and TRQA with 569 questions across 3 subsets. There are five fixed-sample runs. Quality uses a 0–100 scale, with macro-averages over subsets and an overall equal-weight mean of 3 benchmarks. Retrieval and reranking use the top 100 items, returning the top 10 unseen items, with at most 100 turns and an 8192-token input cap; BCP documents are limited to 512 tokens. Nemotron 3 Ultra NVFP4 and MiniMax M3 use high reasoning and M3 trace context. GPT 5.6 Terra and Gemini 3.6 Flash use high reasoning and M2 question context; Sonnet 5 uses adaptive reasoning with M2. Paper-qualified system labels preserve rerankers and modes; these are not scores for unmodified backbones. Piika answer populations are separate, with 6 runs and GPT 5.5 low-effort judging; missing answers receive zero. Metric: Accuracy (%).
Metric: Accuracy (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 15 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | ACARe RetrievalBench GPT-5.6 Terra + ACARe-4B M2 | 44.1 | |
| 2 | ACARe RetrievalBench GPT-5.6 Terra | 43.3 | |
| 3 | ACARe RetrievalBench GPT-5.6 Terra + Qwen3-Reranker-4B | 42.7 | |
| 4 | ACARe RetrievalBench Gemini 3.6 Flash + ACARe-4B M2 | 42.2 | |
| 5 | ACARe RetrievalBench Gemini 3.6 Flash | 40.6 | |
| 6 | ACARe RetrievalBench Gemini 3.6 Flash + Qwen3-Reranker-4B | 40.1 | |
| 7 | ACARe RetrievalBench Claude Sonnet 5 + ACARe-4B M2 | 38.6 | |
| 8 | ACARe RetrievalBench Nemotron-3 Ultra + ACARe-4B M3 | 38.3 | |
| 9 | ACARe RetrievalBench MiniMax M3 + ACARe-4B M3 | 37.4 | |
| 10 | ACARe RetrievalBench Nemotron-3 Ultra | 36.7 |
Interactive version: theaggregate.ai/benchmark?slug=acare-agentic-trqa-ndcg-10 · How It Works · Data refreshed daily, snapshot 2026-10-28.