QuantCode-Bench: leaderboard

QuantCode-Bench (2026) tests whether language models turn textual trading strategy descriptions into Backtrader code that compiles, backtests on historical market data, trades and matches the described strategy. Share of tasks passing every check including an LLM judgment of faithfulness, in percent, from one attempt; higher is better.

Metric: Judge pass rate (%), single turn: share of the tasks whose generated strategy compiles, backtests without runtime errors, places at least one trade and is judged by an LLM to implement the described strategy, on QuantCode-Bench (400 Backtrader trading-strategy tasks from Reddit, TradingView, StackExchange, GitHub and synthetic sources, multi-timeframe historical data); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 17 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Claude Opus 4.675.8#113
2GPT-5.470.2#135
3Claude Sonnet 4.569.8#261
4GPT-5.2 Codex67.5#182
5GLM-565.4#269
6Claude Sonnet 4.665#173
7Kimi K2.564.8#266
8Gemini 3 Flash59.8#175
9Grok 4.1 Fast48.9#427
10DeepSeek V3.248.8#419
11Qwen 3 235B A22B48.2#628
12Qwen 3 Coder 30B A3B Instruct39.2#1012
13Gemini 2.5 Flash31.2#463
14Qwen 3 14B25.2#1269
15Qwen 3 8B18.5#1873

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=quantcode-bench · How It Works · Data refreshed daily, snapshot 2026-11-21.