ACARe ClimbMix57 - Answer Accuracy: leaderboard

ACARe v1 is a fine-tuned Qwen3-Reranker-4B, using full fine-tuning in bf16 on 16,601 groups. RetrievalBench's one-shot and agentic populations use BCP with 200 questions (seed 42, averaging gold and evidence qrels), BRIGHT with 1384 questions across 12 domains, and TRQA with 569 questions across 3 subsets. There are five fixed-sample runs. Quality uses a 0–100 scale, with macro-averages over subsets and an overall equal-weight mean of 3 benchmarks. Retrieval and reranking use the top 100 items, returning the top 10 unseen items, with at most 100 turns and an 8192-token input cap; BCP documents are limited to 512 tokens. Nemotron 3 Ultra NVFP4 and MiniMax M3 use high reasoning and M3 trace context. GPT 5.6 Terra and Gemini 3.6 Flash use high reasoning and M2 question context; Sonnet 5 uses adaptive reasoning with M2. Paper-qualified system labels preserve rerankers and modes; these are not scores for unmodified backbones. Piika answer populations are separate, with 6 runs and GPT 5.5 low-effort judging; missing answers receive zero. This uses all 57 BrowseCompPlus CM questions, a 553M-document corpus, the BM25 API, and 6 runs. It uses high reasoning and a 2-hour timeout. The 19 Lightning 32K reasoning-exhaustion cases count as incorrect and are not removed. Unlabeled documents are treated as nonrelevant; document sets are deduplicated within trajectories. Metric: Accuracy (%).

Metric: Accuracy (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 6 models tracked.

Top models

Interactive version: theaggregate.ai/benchmark?slug=acare-climbmix57-answer-accuracy · How It Works · Data refreshed daily, snapshot 2026-10-28.