RHELM (Long Context) - Aggregation: leaderboard
RHELM category (Zhang et al., Microsoft and Renmin University of China, 2026) that asks for conditional counts, trends, extreme values or events that never happened across many sessions. The model reads the whole history, dialogue and external documents in chronological order, in a 1M-token context; accuracy in percent on 192 questions judged 0 or 1 by an LLM, higher is better.
Metric: Accuracy (%; 192 aggregation questions, LLM judge, dialogue history plus external sources, full history in a 1M-token context). Source: arxiv.org. Saturation forecast: Estimated already saturated. 3 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-4.1 Mini | 33.9 | #775 |
| 2 | Gemini 2.5 Flash Lite | 23.4 | #958 |
| 3 | Qwen2.5-14B-Instruct-1M | 15.6 | #1417 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=rhelm-long-context-aggregation · How It Works · Data refreshed daily, snapshot 2026-11-12.