BAGEN - Warehouse - Interval Reward: leaderboard
BAGEN (arXiv 2606.00198) on a warehouse-operations simulation built from real enterprise data, with three coupled budgets: time (weeks), warehouse occupancy (item-weeks) and cumulative cost (USD). At each prefix of its own logged rollouts on probes whose cash target is reachable, a model that judges the target feasible gives one interval for each remaining resource. Score: mean interval reward (0-1), coverage of the realized remaining amount times the interval's tightness, so a wide safe interval scores low and an impossible verdict scores zero. Unlike cost leaderboards, the model estimates its own consumption before it happens. Each model is scored on its own rollouts. Higher is better. Source: https://arxiv.org/abs/2606.00198
Metric: Mean interval score (coverage x tightness, 0-1). Source: arxiv.org. Saturation forecast: Around February 2027. 5 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) (Low) | 0.7 | #64 (Gemini 3.1 Pro (Preview)) |
| 2 | Claude Opus 4.7 (Low) | 0.69 | #51 (Claude Opus 4.7) |
| 3 | GPT-5.2 (Non-reasoning) | 0.58 | #115 (GPT-5.2) |
| 4 | Claude Sonnet 4.6 (Low) | 0.57 | #94 (Claude Sonnet 4.6) |
| 5 | Qwen 3 235B A22B 2507 Instruct | 0.48 | #315 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=bagen-warehouse-interval-reward · How It Works · Data refreshed daily, snapshot 2026-11-05.