Beyond Continuity: leaderboard
Users interacting with Large Language Models (LLMs) in a multi-turn conversation routinely refine their requests or pivot to new topics. LLMs, however, often miss these topic shifts and carry over irrelevant context from previous turns, leading to inaccurate responses. In this paper, we stress-test the multi-turn understanding of LLMs and study the following two sub-tasks: (1) detecting whether th
Metric: F1 Pivot (TopiOCQA) (self-reported). Source: benchmarklist.com. Saturation forecast: Estimated already saturated. 10 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | O3 | 97.7 | #124 |
| 2 | Claude 3.7 Sonnet | 94.5 | #249 |
| 3 | Gemini 2.5 Pro | 87.9 | #152 |
| 4 | GPT-4o | 70.8 | #343 |
| 5 | Llama 3.1 8B | 69.9 | #1151 |
| 6 | GPT-4o Mini | 64.5 | #599 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=beyond-continuity · How It Works · Data refreshed daily, snapshot 2026-10-20.