ARE-bench - ISR: leaderboard
ARE-bench - ISR: Three-round mean checklist success (%). ARE-bench arXiv 2611.14364v1: 75 mid-session research-continuation tasks, 3 rollouts/task. Overall ISR requires every applicable checklist item, CSR is partial checklist success; values are three-round mean percentages, SD retained in source. GPT-6.1-Sol uses Codex, others Claude Code. Codex 0.118.0 / Claude Code 2.1.283 on RJob, varying by E2B run; Harbor 0.23.0 in 650 readable protocols. GLM historical deployments are mixed in published model aggregates; its aggregate is held for exact per-run checkpoint binding. Qwen3.8-Max retains the archived API label: the bibliography alone does not bind it to the open FP8 checkpoint. Effort is unreported, not inferred from current defaults.
Metric: Three-round mean checklist success (%). Source: arxiv.org. Saturation forecast: Around March 2027. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-6.1 Sol | 66.22 | #10 |
| 2 | Claude Opus 5.5 | 56.89 | #5 |
| 3 | DeepSeek V4.1 Flash | 39.56 | #58 |
| 4 | Qwen 3.8 Max | 37.78 | #55 |
| 5 | Kimi K3 | 28 | #53 |
| 6 | Qwen 3.8 27B | 25.78 | #199 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #6 GPT-6 Astra.
Interactive version: theaggregate.ai/benchmark?slug=are-bench-isr · How It Works · Data refreshed daily, snapshot 2026-11-12.