Retrieval Gate Injection Detection - Q1 AUROC: leaderboard
Released decision models and LLM judges screening retrieved forum-style web passages for hidden instructions aimed at the model. Corpus A: 1,000 questions over forum-style web pages with hidden injected instructions. Each retrieved passage is asked whether it instructs an AI (Q1); AUROC separates retrieved attack passages from all other retrieved passages. Decision models are used as released on one RTX PRO 4500; LLM judges run through OpenRouter and their scores are parsed from JSON. Input is the page HTML or its plain-text chunk; the two inputs stay separate configurations. Rows evaluated on query subsets (Tev1-4B, Von, the Llama-3.1-8B judge) are not admitted, nor the TF-IDF and perplexity baselines, latency or cost. Question access: Corpus release not verified.
Metric: AUROC (0-1). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | DeepSeek V4.1 Flash (Benchmark Sweep Run: source=RetrievalGates; LLM judge via OpenRouter; question Q1; input=HTML; JSON score parsed) | 1 | |
| 2 | Llama 3.3 70B Instruct (Benchmark Sweep Run: source=RetrievalGates; LLM judge via OpenRouter; question Q1; input=HTML; JSON score parsed) | 0.99 | |
| 3 | Kev-4B (Benchmark Sweep Run: source=RetrievalGates; decision model as released; question Q1; input=HTML; no fine-tuning) | 0.94 | |
| 4 | Kev-4B (Benchmark Sweep Run: source=RetrievalGates; decision model as released; question Q1; input=text; no fine-tuning) | 0.87 | |
| 5 | Laya (421M) (Benchmark Sweep Run: source=RetrievalGates; decision model as released; question Q1; input=HTML; no fine-tuning) | 0.75 | |
| 6 | Laya (421M) (Benchmark Sweep Run: source=RetrievalGates; decision model as released; question Q1; input=text; no fine-tuning) | 0.63 |
Interactive version: theaggregate.ai/benchmark?slug=retrieval-gate-injection-detection-q1-auroc · How It Works · Data refreshed daily, snapshot 2026-11-07.