PredicateLongBench - Lexicographic Longest (LongBench): leaderboard
PredicateLongBench asks a model to find the run of words in a very long list that satisfies a stated predicate, such as a lexicographically sorted sequence. This board covers returning the longest sorted run in a LongBench v2 document. Answers are exact-match and outputs that exceed the generation budget count as wrong, so the score reflects both search and output discipline.
Metric: Accuracy (%; share of the 93 LongBench v2 documents under 170K tokens used as word lists whose answer is exactly the one word sequence satisfying the predicate, retrieve the longest lexicographically sorted run; up to 16K output tokens including reasoning, an answer past the cap counts as wrong). Source: arxiv.org. Saturation forecast: Around March 2028. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.6 (Thinking) | 20 | #102 (Claude Opus 4.6) |
| 2 | Gemini 3.1 Pro (Preview) (Low) | 10 | #94 (Gemini 3.1 Pro (Preview)) |
| 3 | GPT-5.4 (High) | 7 | #123 (GPT-5.4) |
| 4 | Claude Opus 4.6 | 5 | #102 |
| 5 | Qwen 3.5 397B A17B | 0 | #246 |
| 6 | GLM-5.1 | 0 | #179 |
| 7 | MiniMax-M2.7 | 0 | #518 |
| 8 | Gemini 3.1 Pro (Preview) (High) | 0 | #94 (Gemini 3.1 Pro (Preview)) |
| 9 | GPT-5.4 (Non-reasoning) | 0 | #123 (GPT-5.4) |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=predicatelongbench-lexicographic-longest-longbench · How It Works · Data refreshed daily, snapshot 2026-11-13.