Rigged Backtest Audit - All-Three Specificity: leaderboard
A clean-control calibration benchmark for LLM audits of trading backtests (Ulesov et al., MBZUAI, 2026): 48 flawed backtests over eight methodology flaws such as look-ahead, survivorship and leakage, each paired with a clean control that changes one detail. Percent of the 48 flawed backtests on which the audit names the right flaw class, cites the planted lines and proposes a class-specific fix, under a prompt warning that clean controls are common and with the backtest given as code; higher is better.
Metric: All-three specificity (%; share of the 48 flawed backtests for which the audit names the right flaw class, cites the planted lines and proposes a class-specific fix; clean-aware prompt on the code surface of a 96-item paired set, 48 flawed backtests over eight flaw classes and 48 matched clean controls, deterministic scorer). Source: arxiv.org. Saturation forecast: Estimated already saturated. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-4.1 Mini | 97.9 | #775 |
| 2 | DeepSeek V4 Flash | 87.5 | #254 |
| 3 | Gemini 2.5 Flash Lite | 85.4 | #958 |
| 4 | GPT-4o Mini | 75 | #1627 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=rigged-backtest-audit-all-three-specificity · How It Works · Data refreshed daily, snapshot 2026-11-12.