Backtrader-Bench (No Tools): leaderboard
Backtrader-Bench (2026) tests LLM coding agents on algorithmic-trading questions whose answers come from executing Backtrader backtests, as four-option multiple choice with chance 25 percent. Accuracy on the curated 30-question set answered from parametric knowledge with tools disabled, averaged over ten runs, in percent; higher is better.
Metric: Accuracy (%; mean of 10 runs on the curated set of 30 four-option multiple-choice questions whose answers come from executing Backtrader backtests of five trading strategies on AAPL daily data 2020-2024 (10 easy, 10 medium, 10 hard); answered from parametric knowledge through the Cursor SDK with every tool disabled; chance 25%). Source: arxiv.org. Saturation forecast: Around February 2027. 10 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) | 73 | #91 |
| 2 | Claude Opus 4.7 | 73 | #75 |
| 3 | GPT-5.3 Codex | 69.3 | #114 |
| 4 | GPT-5.5 | 68 | #48 |
| 5 | Claude Sonnet 4.6 | 67.3 | #158 |
| 6 | Claude Opus 4.6 | 64.3 | #100 |
| 7 | Grok 4.3 | 59.3 | #332 |
| 8 | Kimi K2.5 | 55.7 | #252 |
| 9 | Gemini 2.5 Flash | 49 | #452 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=backtrader-bench-no-tools · How It Works · Data refreshed daily, snapshot 2026-11-12.