Generalization V1 (Lechmazur) — leaderboard
Tests whether LLMs can infer a specific latent theme from limited examples, reject broader but incorrect patterns using anti-examples, and identify the single correct match among similar candidates. V1 uses 810 prompts.
Metric: Avg Rank (lower is better). Source: github.com. Status: saturated. 82 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | Claude Opus 4 (Thinking 16K) | 1.69 |
| 2 | Claude Opus 4.1 (Non-reasoning) | 1.69 |
| 3 | Claude Sonnet 4 (Thinking 16K) | 1.69 |
| 4 | Claude Opus 4 (Non-reasoning) | 1.7 |
| 5 | Gemini 2.5 Pro | 1.74 |
| 6 | DeepSeek R1 0528 | 1.74 |
| 7 | Gemini 2.5 Pro (Preview 05-06) | 1.75 |
| 8 | GPT-5 (Medium) | 1.79 |
| 9 | Grok 4 Fast (Reasoning) | 1.79 |
| 10 | DeepSeek R1 | 1.8 |
| 11 | O1 | 1.8 |
| 12 | GLM-4.5 | 1.8 |
| 13 | O4 Mini (Medium) | 1.8 |
| 14 | O4 Mini (High) | 1.82 |
| 15 | O3 (High) | 1.82 |
Interactive version: theaggregate.ai/benchmark?slug=generalization-v1-lechmazur · How the rankings work · Data refreshed daily, snapshot 2026-07-22.