DriftBench e6 swe cells main / single_shot - Alternative Coverage: leaderboard

DriftBench's camera-ready version is dated 2026-10-26. It uses 38 research briefs across 24 domains with 2 replications; the clean cohort has 2116 runs after 30 stray pilot Sonnet rows are removed. Single-shot runs have 1 turn, neutral/pressure runs have 6 turns, and checkpoint runs have 8 turns. Main output is capped at 4096 tokens and probe output at 1024. GPT and Sonnet use temperature 0.7; Gemini uses temperature 1. Exact Together instruct routes are retained. Opus 4.6 is the common judge for the primary study, GPT 5.4 is the common judge for robustness, and published cross-family judging remains separate. Fidelity, adherence, coverage, and complexity use a 0–4 scale. Knows But Violates (KBV) requires noncompliance (adherence < 4) and final-probe accuracy >= 0.8. Source-specific sample counts, uncapped runs, and judge identities are pinned. This intake makes no model API calls. The SWE pilot is outside the 38 research briefs. The native-execution study of 4 briefs and the main pilot remain separate. This intake reads recorded execution outcomes without executing model code. Metric: Alternative Coverage (0-4).

Metric: Alternative Coverage (0-4). Source: huggingface.co. Saturation forecast: Rough model projection: around 2026. 3 models tracked.

Top models

Interactive version: theaggregate.ai/benchmark?slug=driftbench-e6-swe-cells-main-single-shot-alternative-coverage · How It Works · Data refreshed daily, snapshot 2026-10-28.