BAGEN - SWE-bench - Interval Reward: leaderboard
BAGEN (arXiv 2606.00198) on SWE-bench coding rollouts, remaining cost in tokens: at each prefix of its own successful rollouts a model predicts an interval for the cost still to come. Score: mean interval reward (0-1), coverage of the true remaining cost times the interval's tightness, so a wide safe interval scores low. Unlike cost leaderboards, the model estimates its own spend before it happens. Each model is scored on its own rollouts. Higher is better. Source: https://arxiv.org/abs/2606.00198
Metric: Mean interval score (coverage x tightness, 0-1). Source: arxiv.org. Saturation forecast: Around September 2028. 5 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.7 (Low) | 0.16 | #51 (Claude Opus 4.7) |
| 2 | Claude Sonnet 4.6 (Low) | 0.13 | #94 (Claude Sonnet 4.6) |
| 3 | GPT-5.2 (Non-reasoning) | 0.12 | #115 (GPT-5.2) |
| 4 | Gemini 3.1 Pro (Preview) (Low) | 0.11 | #64 (Gemini 3.1 Pro (Preview)) |
| 5 | Qwen 3 235B A22B 2507 Instruct | 0.02 | #315 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=bagen-swe-bench-interval-reward · How It Works · Data refreshed daily, snapshot 2026-11-05.