LiveBench Summarize: leaderboard
The summarize task in LiveBench's instruction following category: the model is given a recent Guardian article and must summarise it while obeying several automatically checkable instructions, such as word limits. Scoring is adherence to those instructions rather than summary quality, and the board shows the task score in percent, higher is better. Source: arxiv.org/abs/2406.19314.
Metric: Score. Source: livebench.ai. Saturation forecast: Not forecast. 71 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) (High) | 88.23 | #58 (Gemini 3.1 Pro (Preview)) |
| 2 | Gemini 3.8 Flash (High) | 87.5 | #21 (Gemini 3.8 Flash) |
| 3 | Gemini 3.7 Flash (High) | 84.77 | #23 (Gemini 3.7 Flash) |
| 4 | Qwen 3.7 Max | 83.8 | #75 |
| 5 | Muse Spark 1.3 (xHigh) | 83.4 | #24 (Muse Spark 1.3) |
| 6 | Gemini 3.5 Flash (High) | 82.58 | #60 (Gemini 3.5 Flash) |
| 7 | Claude Fable 5 (Max) | 81.7 | #16 (Claude Fable 5) |
| 8 | GPT-6.1 Sol (Max) | 81.25 | #7 (GPT-6.1 Sol) |
| 9 | Muse Spark 1.2 (xHigh) | 80.62 | #62 (Muse Spark 1.2) |
| 10 | GPT-6 Astra (Max) | 79.55 | #6 (GPT-6 Astra) |
| 11 | Qwen 3.8 Max | 79.15 | #35 |
| 12 | Grok 4.7 (xHigh) | 78.17 | #25 (Grok 4.7) |
| 13 | Claude Opus 4.8 (Max) | 77.67 | #37 (Claude Opus 4.8) |
| 14 | Qwen3.8-Flash-Next | 77.35 | #72 |
| 15 | Muse Wonder (Max) | 76.83 | #15 (Muse Wonder) |
No result here: #8 Gemini 4 Argon, #10 Muse Wonder Full.
Interactive version: theaggregate.ai/benchmark?slug=livebench-summarize · How It Works · Data refreshed daily, snapshot 2026-10-20.