GR-Ben (Thinking): leaderboard

GR-Ben (Thinking) scores LLM critics with slow thinking enabled on GR-Ben (2026), where the model must name the earliest erroneous step of a step-by-step scientific or logical solution or state that every step is correct. The score is the average F1 in percent over the nine subsets, F1 being the harmonic mean of accuracy on flawed and on correct solutions, higher is better.

Metric: Average F1 (%) over the nine GR-Ben subsets (four scientific: biology, physics, chemistry and computer science from MMLU-Pro; five logical: abductive, analogical, mixed-form, deductive and inductive), F1 being the harmonic mean of the accuracy on flawed solutions (earliest erroneous step named) and on fully correct solutions; LLMs prompted as critics with slow thinking enabled (GPT-5.2 at xhigh and Gemini 3 Flash at high reasoning effort, 32,768 output tokens); higher is better. Source: arxiv.org. Saturation forecast: Around December 2027. 7 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Gemini 3 FlashHigh61.1#175
2DeepSeek V3.2Thinking56.7#419
3GPT-5.2 (2025-12-11)xHigh54.9
4Qwen 3 32B45.4#972
5Qwen 3 14B42.3#1269
6Qwen 3 8B39.8#1873
7Qwen 3 4B32.5#2348

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=gr-ben-thinking · How It Works · Data refreshed daily, snapshot 2026-11-21.