Lessons Rule Replay - No Fix Em Dash Violations: leaderboard
Lessons Outlive Their Failures paper v1: explicit no-em-dash prose instruction, identical isolated pre-fix repo and cover-letter task,3runs/model/runtime. ConditionA removes mechanical lintfix but retains proseban; native number of committed-output ruleviolations lowerisbetter. All3runs complete for this specific scenario. Private repository/prompts; nofix estimate task omitted because declined/clocksupport differs. Withfix mechanical interceptions are not modelcapability scores. Exact runtime retained; undisclosed Claude effort not inferred. Metric: Instruction violation count (3 runs).
Metric: Instruction violation count (3 runs). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 5 (Harness: Lessons study Claude Code) | 0 | |
| 2 | Claude Sonnet 5.5 (Harness: Lessons study Claude Code) | 0 | |
| 3 | Gemini 3.8 Flash (medium) (Harness: Lessons study Antigravity) | 0 | |
| 4 | Claude Sonnet 4.5 (Harness: Lessons study Claude Code) | 1 |
Interactive version: theaggregate.ai/benchmark?slug=lessons-rule-replay-no-fix-em-dash-violations · How It Works · Data refreshed daily, snapshot 2026-10-27.