NovBench - Correctness: leaderboard

NovBench (2026) tests whether LLMs can write the novelty assessment of a research paper from the novelty claims in its introduction, on 1,684 EMNLP 2023 papers with expert peer reviews. Correctness is the agreement between the positive, neutral and negative balance of the generated statements and that of the human reviewers, from 0 to 1, higher is better.

Metric: Correctness (DistAcc, 0-1): one minus half the L1 distance between the positive, neutral and negative proportions of the generated statements and of the human comments; zero-shot prompt; 1,684 EMNLP 2023 papers: from the novelty-description sentences of the paper introduction (selected by GPT-5) the model writes a novelty evaluation as positive, neutral and negative statements, compared with the expert reviewers novelty comments (extracted by GPT-4o-mini, then deduplicated, consolidated and sorted by sentiment by GPT-4o); greedy decoding, at most 4,096 tokens; higher is better. Source: arxiv.org. Saturation forecast: Not forecast. 18 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1SEA-S0.7162
2SEA-E0.6834
3Qwen 3 14B0.6627#1269
4Qwen 3 8B0.6595#1873
5DeepSeek R1 Distill Llama 70B0.6572#1669
6Qwen 3 32B0.6555#972
7DeepSeek R1 Distill Qwen 14B0.6436#2771
8DeepReviewer-14B0.6356#844
9CycleReviewer-70B0.6326#1880
10DeepReviewer-7B0.6134
11Gemini 2.5 Flash0.6011#463
12GPT-OSS-20B0.5785#1183
13DeepSeek-R1-8B (NovBench checkpoint unspecified)0.5143
14GPT-50.483#178
15Reviewer20.4517

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=novbench-correctness · How It Works · Data refreshed daily, snapshot 2026-11-21.