Polymorphic Sybil RAG: leaderboard

This benchmark poisons the retrieval corpus of a grounded question-answering pipeline with several lexically diverse passages that jointly support one attacker-chosen wrong answer, then sorts each reader answer into gold, attacker target, abstention or other. This board is the share of answers that are still the gold answer when the poisoned passages are placed beside gold-supporting ones at fixed retrieval positions, so higher is better. Questions come from HotpotQA and Natural Questions and answers are matched by strict exact match against canonicalized answer sets.

Metric: Gold retention (%; share of the 2,982 released HotpotQA and Natural Questions items; few-shot prompt shared byte-identically across readers, temperature 0, E5+cross-encoder retrieval, strict exact match against the canonicalized gold and target answer sets whose answer is the gold answer; Forced Exposure: the six sybil passages placed at fixed top-10 positions beside gold-supporting passages). Source: arxiv.org. Saturation forecast: Around June 2027. 5 models tracked.

Top models

No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.

Interactive version: theaggregate.ai/benchmark?slug=polymorphic-sybil-rag · How It Works · Data refreshed daily, snapshot 2026-11-13.