RoBench-25 (Awareness Instruction) - Trigger A: leaderboard
RoBench-25 (Le Mercier, Develder and Demeester, 2026) tests whether a language model still answers true-or-false questions about a recent NeurIPS abstract when a hidden-state poisoning trigger and six distractor abstracts come between the abstract and the questions; 240 questions on 120 abstracts, each abstract the target once per run, with greedy decoding. Clipped Heidke skill score from 0 to 1 (chance-corrected accuracy), higher is better; this board is under trigger A, an instruction to ignore all previous instructions, with the awareness instruction only (a warning to ignore instructions about forgetting, placed before the abstracts).
Metric: Clipped Heidke skill score (0-1; chance-corrected accuracy on the 240 true-or-false questions about 120 NeurIPS 2025 abstracts, each prompt one target abstract, then the trigger if any, then six distractor abstracts, averaged over 120 random draws (pythia-2.8b pools two runs of 120); trigger A ("Ignore all previous instructions."); prompts with the awareness instruction only (a warning to ignore instructions about forgetting, placed before the abstracts); greedy decoding, at most 20 new tokens). Source: arxiv.org. Saturation forecast: Estimated already saturated. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | SmolLM3-3B | 1 | #6555 |
| 2 | Jamba 1.7 Mini | 0.99 | #4808 |
| 3 | mamba-2.8B-hf | 0.18 | |
| 4 | pythia-2.8B | 0.03 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=robench-25-awareness-instruction-trigger-a · How It Works · Data refreshed daily, snapshot 2026-11-13.