BIG-Bench Extra Hard: leaderboard
23 reasoning tasks (4,520 items) from Google DeepMind (2025), each a harder replacement for a BIG-Bench Hard task; harmonic mean of per-task accuracy, 9.8% for the best general model at release.
Metric: Score (imported). Source: arxiv.org. Source: benchmarklist.com. 17 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemma 4 31B | 74.4 | #200 |
| 2 | Gemma 4 26B A4B | 64.8 | #358 |
| 3 | Gemma 4 12B Unified | 53 | |
| 4 | o3-mini (high) | 44.8 | |
| 5 | Gemma 4 E4B | 33.1 | #767 |
| 6 | Gemma 4 E2B | 21.9 | #1070 |
| 7 | Gemini 2.0 Flash | 9.8 | #342 |
| 8 | Gemini 2.0 Flash-Lite | 8 | |
| 9 | DeepSeek R1 | 6.8 | #251 |
| 10 | GPT4o | 6 |
Interactive version: theaggregate.ai/benchmark?slug=big-bench-extra-hard · How It Works · Data refreshed daily, snapshot 2026-10-20.