PredicateLongBench - Lexicographic Locate (LongBench): leaderboard
PredicateLongBench asks a model to find the run of words in a very long list that satisfies a stated predicate, such as a lexicographically sorted sequence. This board covers locating the longest sorted run in a document drawn from LongBench v2. Answers are exact-match and outputs that exceed the generation budget count as wrong, so the score reflects both search and output discipline.
Metric: Accuracy (%; share of the 93 LongBench v2 documents under 170K tokens used as word lists whose answer is exactly the one word sequence satisfying the predicate, locate the lexicographically sorted run as long as the longest in the document, no decoys; up to 16K output tokens including reasoning, an answer past the cap counts as wrong). Source: arxiv.org. Saturation forecast: Around February 2027. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.6 (Thinking) | 61 | #102 (Claude Opus 4.6) |
| 2 | GPT-5.4 (High) | 36 | #123 (GPT-5.4) |
| 3 | Claude Opus 4.6 | 10 | #102 |
| 4 | Gemini 3.1 Pro (Preview) (Low) | 10 | #94 (Gemini 3.1 Pro (Preview)) |
| 5 | Gemini 3.1 Pro (Preview) (High) | 6 | #94 (Gemini 3.1 Pro (Preview)) |
| 6 | GLM-5.1 | 3 | #179 |
| 7 | GPT-5.4 (Non-reasoning) | 1 | #123 (GPT-5.4) |
| 8 | Qwen 3.5 397B A17B | 0 | #246 |
| 9 | MiniMax-M2.7 | 0 | #518 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=predicatelongbench-lexicographic-locate-longbench · How It Works · Data refreshed daily, snapshot 2026-11-13.