Rigged Backtest Audit - Clean-Control False Positive Rate: leaderboard
A clean-control calibration benchmark for LLM audits of trading backtests (Ulesov et al., MBZUAI, 2026): 48 flawed backtests over eight methodology flaws such as look-ahead, survivorship and leakage, each paired with a clean control that changes one detail. Percent of the 48 clean controls wrongly flagged as flawed, under a prompt warning that clean controls are common and with the backtest given as code; lower is better.
Metric: False positive rate on clean controls (%; share of the 48 clean backtests the model flags as flawed; clean-aware prompt on the code surface of a 96-item paired set, 48 flawed backtests over eight flaw classes and 48 matched clean controls, deterministic scorer). Source: arxiv.org. Saturation forecast: Estimated already saturated. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | DeepSeek V4 Flash | 0 | #254 |
| 2 | Gemini 2.5 Flash Lite | 0 | #958 |
| 3 | GPT-4.1 Mini | 16.7 | #775 |
| 4 | GPT-4o Mini | 79.2 | #1627 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=rigged-backtest-audit-clean-control-false-positive-rate · How It Works · Data refreshed daily, snapshot 2026-11-12.