PredicateLongBench - Unary For-All: leaderboard
PredicateLongBench asks a model to find the run of words in a very long list that satisfies a stated predicate, such as a lexicographically sorted sequence. This board covers the same unary predicate asked with a universal quantifier, so every sequence must be checked. Answers are exact-match and outputs that exceed the generation budget count as wrong, so the score reflects both search and output discipline.
Metric: Accuracy (%; share of the 100 synthetic word lists of about 128K tokens whose answer is exactly the one word sequence satisfying the predicate, retrieve every matching run when exactly one exists (universal quantifier); up to 16K output tokens including reasoning, an answer past the cap counts as wrong). Source: arxiv.org. Saturation forecast: Estimated already saturated. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.6 (Thinking) | 99 | #102 (Claude Opus 4.6) |
| 2 | GPT-5.4 (High) | 93 | #123 (GPT-5.4) |
| 3 | GPT-5.4 (Non-reasoning) | 93 | #123 (GPT-5.4) |
| 4 | Claude Opus 4.6 | 90 | #102 |
| 5 | GLM-5.1 | 73 | #179 |
| 6 | Gemini 3.1 Pro (Preview) (High) | 59 | #94 (Gemini 3.1 Pro (Preview)) |
| 7 | Qwen 3.5 397B A17B | 43 | #246 |
| 8 | MiniMax-M2.7 | 30 | #518 |
| 9 | Gemini 3.1 Pro (Preview) (Low) | 20 | #94 (Gemini 3.1 Pro (Preview)) |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=predicatelongbench-unary-for-all · How It Works · Data refreshed daily, snapshot 2026-11-13.