Kaggle Math Puzzles (Lewis Carroll Sorites): leaderboard
Community benchmark on Kaggle by junesdata: 16 sorites from Lewis Carroll's Symbolic Logic (1896), each asking for the single conclusion that follows from 3 to 10 premises. A puzzle passes when the reply names the required terms and a judge model accepts the conclusion and its reasoning chain; percent of puzzles passed, higher is better. The puzzles and their solutions are public.
Metric: Puzzles Passed (%). Source: www.kaggle.com. Saturation forecast: Not forecast. 12 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GLM-5 | 100 | #149 |
| 2 | Gemini 2.5 Flash | 87.5 | #255 |
| 3 | Gemini 3.5 Flash | 87.5 | #66 |
| 4 | GPT-6 Astra | 87.5 | #6 |
| 5 | Gemini 3.6 Flash | 87.5 | #57 |
| 6 | DeepSeek R1 0528 | 81.25 | #232 |
| 7 | Gemma 4 26B A4B | 68.75 | #360 |
| 8 | Grok 4.20 (Reasoning) | 68.75 | #179 (Grok 4.20) |
| 9 | Claude Opus 4.7 | 56.25 | #51 |
| 10 | Claude Sonnet 4.5 | 43.75 | #148 |
| 11 | GPT-5.4 | 43.75 | #83 |
No result here: #3 Claude Opus 5.5, #7 GPT-6.1 Sol, #9 Gemini 4 Argon.
Interactive version: theaggregate.ai/benchmark?slug=kaggle-math-puzzles-lewis-carroll-sorites · How It Works · Data refreshed daily, snapshot 2026-11-05.