ACARe One-Shot BRIGHT Biology - Recall@10: leaderboard

ACARe v1 is a fine-tuned Qwen3-Reranker-4B, using full fine-tuning in bf16 on 16,601 groups. RetrievalBench's one-shot and agentic populations use BCP with 200 questions (seed 42, averaging gold and evidence qrels), BRIGHT with 1384 questions across 12 domains, and TRQA with 569 questions across 3 subsets. There are five fixed-sample runs. Quality uses a 0–100 scale, with macro-averages over subsets and an overall equal-weight mean of 3 benchmarks. Retrieval and reranking use the top 100 items, returning the top 10 unseen items, with at most 100 turns and an 8192-token input cap; BCP documents are limited to 512 tokens. Nemotron 3 Ultra NVFP4 and MiniMax M3 use high reasoning and M3 trace context. GPT 5.6 Terra and Gemini 3.6 Flash use high reasoning and M2 question context; Sonnet 5 uses adaptive reasoning with M2. Paper-qualified system labels preserve rerankers and modes; these are not scores for unmodified backbones. Piika answer populations are separate, with 6 runs and GPT 5.5 low-effort judging; missing answers receive zero. Metric: Recall@10 (%).

Metric: Recall@10 (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 16 models tracked.

Top models

#ModelScoreOverall rank
1ACARe one-shot AgentIR-4B + ACARe-4B M044.1
2ACARe one-shot Nemotron-3-Embed-8B + ACARe-4B M043.8
3ACARe one-shot Qwen3-Embedding-4B + ACARe-4B M040.3
4ACARe one-shot Nemotron-3-Embed-8B39.1
5ACARe one-shot Nemotron-3-Embed-8B + Qwen3-Reranker-4B37.5
6ACARe one-shot AgentIR-4B + Qwen3-Reranker-4B36.4
7ACARe one-shot Qwen3-Embedding-4B + Qwen3-Reranker-4B35.7
8ACARe one-shot AgentIR-4B + Jina-Reranker-v332.8
9ACARe one-shot BM25 + ACARe-4B M031.1
10ACARe one-shot AgentIR-4B29.7

Interactive version: theaggregate.ai/benchmark?slug=acare-one-shot-bright-biology-recall-10 · How It Works · Data refreshed daily, snapshot 2026-10-28.