DriftBench probe accuracy summary checkpointed_pressure - Banned Move Recall: leaderboard

DriftBench's camera-ready version is dated 2026-10-26. It uses 38 research briefs across 24 domains with 2 replications; the clean cohort has 2116 runs after 30 stray pilot Sonnet rows are removed. Single-shot runs have 1 turn, neutral/pressure runs have 6 turns, and checkpoint runs have 8 turns. Main output is capped at 4096 tokens and probe output at 1024. GPT and Sonnet use temperature 0.7; Gemini uses temperature 1. Exact Together instruct routes are retained. Opus 4.6 is the common judge for the primary study, GPT 5.4 is the common judge for robustness, and published cross-family judging remains separate. Fidelity, adherence, coverage, and complexity use a 0–4 scale. Knows But Violates (KBV) requires noncompliance (adherence < 4) and final-probe accuracy >= 0.8. Source-specific sample counts, uncapped runs, and judge identities are pinned. This intake makes no model API calls. These are primary clean-probe aggregates. Published stray pilots and postselected untruncated rows remain diagnostics, separate from the primary comparison. Metric: Banned Move Recall (fraction).

Metric: Banned Move Recall (fraction). Source: huggingface.co. Saturation forecast: Rough model projection: around 2026. 7 models tracked.

Top models

Interactive version: theaggregate.ai/benchmark?slug=driftbench-probe-accuracy-summary-checkpointed-pressure-banned-move-recall · How It Works · Data refreshed daily, snapshot 2026-10-28.