BIG-Bench Extra Hard: leaderboard
23 reasoning tasks (4,520 items) from Google DeepMind (2025), each a harder replacement for a BIG-Bench Hard task; harmonic mean of per-task accuracy, 9.8% for the best general model at release.
Metric: Score (self-reported). Source: arxiv.org. 17 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | Gemma 4 31B | 74.4 |
| 2 | Gemma 4 26B A4B | 64.8 |
| 3 | Gemma 4 12B Unified | 53 |
| 4 | o3-mini (high) | 44.8 |
| 5 | Gemma 4 E4B | 33.1 |
| 6 | Gemma 4 E2B | 21.9 |
| 7 | Gemini 2.0 Flash | 9.8 |
| 8 | Gemini 2.0 Flash-Lite | 8 |
| 9 | DeepSeek R1 | 6.8 |
| 10 | GPT4o | 6 |
Interactive version: theaggregate.ai/benchmark?slug=big-bench-extra-hard · How It Works · Data refreshed daily, snapshot 2026-09-05.