StakeBench - Commitment Detection: leaderboard
Market commitment detection in StakeBench (2026): given a prediction-market comment, decide whether its author holds a position in that market, using Polymarket and Manifold records rather than annotator labels. The score is accuracy in percent, macro-averaged over topic and platform splits; higher is better.
Metric: Accuracy (%) on G1: does a comment come from a user holding a position in the market (yes or no, chance 50), one-shot prompt, Polymarket and Manifold comments of resolved markets, macro-average over 18 topic and platform splits, greedy decoding; higher is better. Source: arxiv.org. Saturation forecast: Around December 2027. 15 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Qwen 3 32B | Non-reasoning | 74.7 | #972 |
| 2 | Qwen 3 14B | Non-reasoning | 74 | #1269 |
| 3 | DeepSeek R1 Distill Llama 8B | 72.1 | #7059 | |
| 4 | Gemma 2 9B | 71.8 | #3201 | |
| 5 | Gemini 2.5 Flash | 71.1 | #463 | |
| 6 | Claude Haiku 4.5 | 69.7 | #583 | |
| 7 | Qwen 3 8B | Non-reasoning | 69.2 | #1873 |
| 8 | GPT-5.5 | 68.9 | #54 | |
| 9 | Qwen 3 30B A3B | Non-reasoning | 67 | #1156 |
| 10 | Fino1-14B | 64.8 | ||
| 11 | Fino1-8B | 60.1 | #6784 | |
| 12 | Llama 3.2 3B Instruct | 56.7 | #7241 | |
| 13 | Mistral 7B Instruct (v0.3) | 53.4 | #6750 | |
| 14 | Fin-Chat-7B | 53.1 | ||
| 15 | FinMA-30B | 50.3 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=stakebench-commitment-detection · How It Works · Data refreshed daily, snapshot 2026-11-21.