ACARe BCP Context Ablation - Recall@10: leaderboard

ACARe v1 is a fine-tuned Qwen3-Reranker-4B, using full fine-tuning in bf16 on 16,601 groups. RetrievalBench's one-shot and agentic populations use BCP with 200 questions (seed 42, averaging gold and evidence qrels), BRIGHT with 1384 questions across 12 domains, and TRQA with 569 questions across 3 subsets. There are five fixed-sample runs. Quality uses a 0–100 scale, with macro-averages over subsets and an overall equal-weight mean of 3 benchmarks. Retrieval and reranking use the top 100 items, returning the top 10 unseen items, with at most 100 turns and an 8192-token input cap; BCP documents are limited to 512 tokens. Nemotron 3 Ultra NVFP4 and MiniMax M3 use high reasoning and M3 trace context. GPT 5.6 Terra and Gemini 3.6 Flash use high reasoning and M2 question context; Sonnet 5 uses adaptive reasoning with M2. Paper-qualified system labels preserve rerankers and modes; these are not scores for unmodified backbones. Piika answer populations are separate, with 6 runs and GPT 5.5 low-effort judging; missing answers receive zero. The no-reranker, Qwen3 reranker, and M3 arms repeat the main BCP results. Only M0/M1/M2 are additional arms. Metric: Accuracy (%).

Metric: Accuracy (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 3 models tracked.

Top models

Interactive version: theaggregate.ai/benchmark?slug=acare-bcp-context-ablation-recall-10 · How It Works · Data refreshed daily, snapshot 2026-10-28.