DriftBench e7 strict probe accuracy - Strict Constraint Recall: leaderboard

DriftBench's camera-ready version is dated 2026-10-26. It uses 38 research briefs across 24 domains with 2 replications; the clean cohort has 2116 runs after 30 stray pilot Sonnet rows are removed. Single-shot runs have 1 turn, neutral/pressure runs have 6 turns, and checkpoint runs have 8 turns. Main output is capped at 4096 tokens and probe output at 1024. GPT and Sonnet use temperature 0.7; Gemini uses temperature 1. Exact Together instruct routes are retained. Opus 4.6 is the common judge for the primary study, GPT 5.4 is the common judge for robustness, and published cross-family judging remains separate. Fidelity, adherence, coverage, and complexity use a 0–4 scale. Knows But Violates (KBV) requires noncompliance (adherence < 4) and final-probe accuracy >= 0.8. Source-specific sample counts, uncapped runs, and judge identities are pinned. This intake makes no model API calls. Strict probe scoring covers 4 selected models, using the native Opus 4.6 grader and 76 pressure probes per model. It remains separate from April’s lenient grading. Metric: Strict Constraint Recall (fraction).

Metric: Strict Constraint Recall (fraction). Source: huggingface.co. Saturation forecast: Rough model projection: around 2026. 4 models tracked.

Top models

Interactive version: theaggregate.ai/benchmark?slug=driftbench-e7-strict-probe-accuracy-strict-constraint-recall · How It Works · Data refreshed daily, snapshot 2026-10-28.