ACARe Piika BCP200 - Recall@all: leaderboard
ACARe v1 is a fine-tuned Qwen3-Reranker-4B, using full fine-tuning in bf16 on 16,601 groups. RetrievalBench's one-shot and agentic populations use BCP with 200 questions (seed 42, averaging gold and evidence qrels), BRIGHT with 1384 questions across 12 domains, and TRQA with 569 questions across 3 subsets. There are five fixed-sample runs. Quality uses a 0–100 scale, with macro-averages over subsets and an overall equal-weight mean of 3 benchmarks. Retrieval and reranking use the top 100 items, returning the top 10 unseen items, with at most 100 turns and an 8192-token input cap; BCP documents are limited to 512 tokens. Nemotron 3 Ultra NVFP4 and MiniMax M3 use high reasoning and M3 trace context. GPT 5.6 Terra and Gemini 3.6 Flash use high reasoning and M2 question context; Sonnet 5 uses adaptive reasoning with M2. Paper-qualified system labels preserve rerankers and modes; these are not scores for unmodified backbones. Piika answer populations are separate, with 6 runs and GPT 5.5 low-effort judging; missing answers receive zero. This uses a stratified BCP sample with seed 42 and 200 questions, separate from RetrievalBench, with 6 runs per arm. Answers missing due to timeouts, API errors, or token exhaustion receive zero. There are 30 runs and 6000 executions. Metric: Accuracy (%).
Metric: Accuracy (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 5 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | ACARe Piika Lightning BM25 ACARe-4B M3 | 65.1 | |
| 2 | ACARe Piika Lightning BM25 Qwen3-Reranker-4B M3 | 63.9 | |
| 3 | ACARe Piika Lightning BM25 Qwen3-Reranker-4B M2 | 61.2 | |
| 4 | ACARe Piika Lightning BM25 Qwen3-Reranker-4B M0 | 54.2 | |
| 5 | ACARe Piika Lightning BM25 No reranker none | 44.3 |
Interactive version: theaggregate.ai/benchmark?slug=acare-piika-bcp200-recall-all · How It Works · Data refreshed daily, snapshot 2026-10-28.