MintEval (200-Task Subset): leaderboard
MintEval v0 (arXiv 2610.03080): the model turns a trader's colloquial instruction into a trading-strategy program, which is executed bar by bar against the reference program on identical BTCUSDT 15-minute data and frictions. ActionMatch is the share of active bars (either program in or targeting a position) on which the two programs target the same position; this board uses the all-task ActionMatch_all0 variant, assigning compile failures 0, while the paper’s headline ActionMatch averages only compiled runs. Open setting (instruction and interface only), temperature 0, on the stratified 200-task subset (40 per state-span quintile) that the frontier models ran. Percent, higher is better.
Metric: ActionMatch_all0 (%), open setting, compile failures scored 0, 200 stratified BTCUSDT tasks. Source: github.com. Saturation forecast: Around February 2027. 4 models tracked. Updated 14 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Claude Opus 5.5 | 88.9 | #9 | |
| 2 | Qwen 3.8 Max | Low | 81.9 | #66 |
| 3 | DeepSeek V4 Pro | High | 71.2 | #198 |
| 4 | GPT-5.4 Mini | 54.3 | #405 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=minteval-200-task-subset · How It Works · Data refreshed daily, snapshot 2026-11-20.