AGGBench: leaderboard
AGGBench (Ao et al., 2026) tests completeness-oriented aggregation querying over a 45-paper text corpus with extreme evidence sparsity: given a natural-language query with compositional conditions, a system must find every entity in the corpus that satisfies them, across 362 queries with answer sets up to 29 entities. This board runs the paper's DFA agentic pipeline over four base language models and scores Normalized Absolute Count Error, the absolute difference between the predicted and gold entity counts divided by the gold count, so lower is better and values can exceed one.
Metric: Normalized Absolute Count Error (mean; |predicted count - gold count| / gold count over 362 aggregation queries, DFA pipeline). Source: arxiv.org. Saturation forecast: Around February 2027. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3 Flash | 0.73 | #159 |
| 2 | GPT-4o Mini | 0.86 | #1627 |
| 3 | Qwen 3 32B | 1.73 | #960 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=aggbench · How It Works · Data refreshed daily, snapshot 2026-11-12.