SysTradeBench - Execution Gate: leaderboard

Execution gate of SysTradeBench (2026): for 12 standardized trading-strategy specifications the model writes, zero-shot, a strategy card, trading-system code and audit logging, and the gate counts the strategies whose code runs in the sandboxed backtest harness. Pass rate in percent; higher is better.

Metric: Execution gate pass rate (%): share of the 12 strategies whose generated trading-system code executes in the sandboxed harness, one of six validity gates applied in order to each zero-shot (Iter0, temperature 0) submission of strategy card, executable code and audit log for 12 standardized trading-strategy specifications: parse, schema, execution, determinism, anti-leakage and audit; higher is better. Source: arxiv.org. Saturation forecast: Not forecast. 17 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-5.2100#185
2GPT-5.1100#240
3O3100#237
4Grok 4 Fast100#447
5Claude Sonnet 4.591.7#261
6GLM-4.691.7#498
7Grok 4.1 FastReasoning91.7#427
8SysTradeBench DeepSeek-V3 (checkpoint unspecified)91.7
9Claude Opus 4.583.3#160
10Grok 383.3#611
11Grok Code Fast 183.3#703
12Gemini 3 Pro75#145
13GLM-4.758.3#380
14Gemini 2.5 Pro50#276
15Gemini 3 Flash41.7#175

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=systradebench-execution-gate · How It Works · Data refreshed daily, snapshot 2026-11-21.