AutoRace: leaderboard
Six reasoning tasks across maths, logic, commonsense and embodied planning, scored on the reasoning chain rather than the final answer — tasks without an oracle checker are graded by AutoRace, an LLM judge built for chain evaluation. Roster is the GPT-4-turbo / Claude 3 Opus generation.
Metric: Average Score (%, six reasoning tasks). Source: www.llm-reasoners.net. Saturation forecast: Estimated already saturated. 10 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-4 Turbo | 61 | #404 |
| 2 | Claude 3 Opus | 60 | #571 |
| 3 | Gemini 1.0 Pro | 36 | #933 |
| 4 | Mixtral 8x7B | 27 | #1143 |
| 5 | Mistral 7B | 26 | #1397 |
| 6 | Llama 2 70B | 24 | #1201 |
| 7 | gemma-7B | 23 | #1384 |
| 8 | Qwen 1.5 7B | 22 | #1327 |
| 9 | Llama 2 13B | 18 | #1436 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=autorace · How It Works · Data refreshed daily, snapshot 2026-10-20.