ARCBench S5 - Legacy Injection: leaderboard
ARCBench pi agent harness; same exact endpoint is agent and compactor, 17 task cases with only configurations having all valid repetitions included. Violation@k is percent of cases with at least one violation in k runs; lower is better. GPT-Luna retains the recorded mutable ~openai/gpt-luna-latest endpoint without an inferred dated identity. S5; k=3, channels c1-c4, Hermes compaction, Legacy Injection; no defense. Baseline without added content is retained once in S1.
Metric: Violation@3 (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | deepseek-v4-flash-0731 (ARCBench Run: endpoint=deepseek%2Fdeepseek-v4-flash-0731; harness=pi; strategy=Hermes; attack=Legacy%20Injection) | 32.4 | |
| 2 | glm-5.3-flash (ARCBench Run: endpoint=z-ai%2Fglm-5.3-flash; harness=pi; strategy=Hermes; attack=Legacy%20Injection) | 33.3 | |
| 3 | gpt-luna-latest (ARCBench Run: endpoint=~openai%2Fgpt-luna-latest; harness=pi; strategy=Hermes; attack=Legacy%20Injection) | 45.5 | |
| 4 | qwen3.7-flash (ARCBench Run: endpoint=qwen%2Fqwen3.7-flash; harness=pi; strategy=Hermes; attack=Legacy%20Injection) | 73.5 |
Interactive version: theaggregate.ai/benchmark?slug=arcbench-s5-legacy-injection · How It Works · Data refreshed daily, snapshot 2026-11-05.