OracleProto: leaderboard
Forecasting benchmark (OracleProto, Ma et al., Beijing University of Posts and Telecommunications, 2026) that replays 80 resolved FutureX-Past questions, all resolved after every tested model's training cutoff, to a ReAct agent whose only tool is a Tavily web search limited to pages published before the day before resolution, with an LLM leak detector on every result. Each question is answered three times; the score is composite accuracy in percent, exam-style partial credit (zero for any wrong option chosen) weighted 0.15 on yes/no, 0.15 on binary and 0.70 on multiple-choice questions; higher is better.
Metric: Composite accuracy (%; 80 time-masked forecasting questions, exam-style partial credit weighted 0.15 yes/no, 0.15 binary, 0.70 multiple choice, mean of 3 samples). Source: arxiv.org. Saturation forecast: Estimated already saturated. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | DeepSeek V3.2 Exp | 60.16 | #436 |
| 2 | GLM-5 | 60.02 | #256 |
| 3 | Qwen 3.5 Flash | 58.96 | #443 |
| 4 | Kimi K2.5 | 58 | #252 |
| 5 | MiniMax-M2.5 | 54.94 | #590 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=oracleproto · How It Works · Data refreshed daily, snapshot 2026-11-12.