HELM NaturalQuestions (Open): leaderboard

Open-book question answering from Google search queries with access to Wikipedia passages, scored by F1. Part of Stanford HELM.

Metric: F1 (%). Source: crfm.stanford.edu. Saturation forecast: Estimated already saturated. 91 models tracked.

Top models

#ModelScoreOverall rank
1Nova Pro82.9#595
2Nova Lite81.52#742
3GPT-4o (2024-05-13)80.32#337
4GPT-4 Turbo79.52#418
5GPT-4o (2024-08-06)79.26#343
6GPT-4 (0613)78.97#495
7Nova Micro77.86#891
8Qwen 1.5 32B77.71#793
9Command77.69#1297
10Qwen 2 72B Instruct77.58#580
11Yi 34B (Base)77.51#952
12Qwen 1.5 14B77.21#1082
13text-davinci-00376.98#1107
14DeepSeek V376.53#333
15GPT-4 Turbo (Preview)76.29#417

No result here: #3 Claude Opus 5.5, #5 GPT-6 Astra, #6 GPT-6.1 Sol.

Interactive version: theaggregate.ai/benchmark?slug=helm-naturalquestions-open · How It Works · Data refreshed daily, snapshot 2026-11-06.