ACARe BRIGHT Retriever Control - Recall@10: leaderboard
ACARe v1 is a fine-tuned Qwen3-Reranker-4B, using full fine-tuning in bf16 on 16,601 groups. RetrievalBench's one-shot and agentic populations use BCP with 200 questions (seed 42, averaging gold and evidence qrels), BRIGHT with 1384 questions across 12 domains, and TRQA with 569 questions across 3 subsets. There are five fixed-sample runs. Quality uses a 0–100 scale, with macro-averages over subsets and an overall equal-weight mean of 3 benchmarks. Retrieval and reranking use the top 100 items, returning the top 10 unseen items, with at most 100 turns and an 8192-token input cap; BCP documents are limited to 512 tokens. Nemotron 3 Ultra NVFP4 and MiniMax M3 use high reasoning and M3 trace context. GPT 5.6 Terra and Gemini 3.6 Flash use high reasoning and M2 question context; Sonnet 5 uses adaptive reasoning with M2. Paper-qualified system labels preserve rerankers and modes; these are not scores for unmodified backbones. Piika answer populations are separate, with 6 runs and GPT 5.5 low-effort judging; missing answers receive zero. The first stage uses Qwen3 Embedding 4B. Nemotron Embed rows duplicate Table 13. Nemotron Embed training uses the BRIGHT seed corpus, so this is not presented as a clean holdout. Metric: Accuracy (%).
Metric: Accuracy (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 3 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | ACARe Nemotron3UltraNVFP4 RetrievalBench Qwen3-Embedding-4B + ACARe-4B M3 | 47 | |
| 2 | ACARe Nemotron3UltraNVFP4 RetrievalBench Qwen3-Embedding-4B + Qwen3-Reranker-4B | 45.9 | |
| 3 | ACARe Nemotron3UltraNVFP4 RetrievalBench Qwen3-Embedding-4B | 44.2 |
Interactive version: theaggregate.ai/benchmark?slug=acare-bright-retriever-control-recall-10 · How It Works · Data refreshed daily, snapshot 2026-10-28.