SysTradeBench - Execution Gate: leaderboard
Execution gate of SysTradeBench (2026): for 12 standardized trading-strategy specifications the model writes, zero-shot, a strategy card, trading-system code and audit logging, and the gate counts the strategies whose code runs in the sandboxed backtest harness. Pass rate in percent; higher is better.
Metric: Execution gate pass rate (%): share of the 12 strategies whose generated trading-system code executes in the sandboxed harness, one of six validity gates applied in order to each zero-shot (Iter0, temperature 0) submission of strategy card, executable code and audit log for 12 standardized trading-strategy specifications: parse, schema, execution, determinism, anti-leakage and audit; higher is better. Source: arxiv.org. Saturation forecast: Not forecast. 17 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | GPT-5.2 | 100 | #185 | |
| 2 | GPT-5.1 | 100 | #240 | |
| 3 | O3 | 100 | #237 | |
| 4 | Grok 4 Fast | 100 | #447 | |
| 5 | Claude Sonnet 4.5 | 91.7 | #261 | |
| 6 | GLM-4.6 | 91.7 | #498 | |
| 7 | Grok 4.1 Fast | Reasoning | 91.7 | #427 |
| 8 | SysTradeBench DeepSeek-V3 (checkpoint unspecified) | 91.7 | ||
| 9 | Claude Opus 4.5 | 83.3 | #160 | |
| 10 | Grok 3 | 83.3 | #611 | |
| 11 | Grok Code Fast 1 | 83.3 | #703 | |
| 12 | Gemini 3 Pro | 75 | #145 | |
| 13 | GLM-4.7 | 58.3 | #380 | |
| 14 | Gemini 2.5 Pro | 50 | #276 | |
| 15 | Gemini 3 Flash | 41.7 | #175 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=systradebench-execution-gate · How It Works · Data refreshed daily, snapshot 2026-11-21.