JudgeSense - Factuality: leaderboard

JudgeSense (2026) measures how often an LLM judge gives the same verdict when one judging request is worded two equivalent ways, at temperature 0, on items from human-labelled corpora. Agreement across the two wordings when judging TruthfulQA statements as accurate or inaccurate, from 0 to 1; higher is better.

Metric: Judge Sensitivity Score: share of items on which the judge returns the same verdict under two reworded instruction templates that ask for the same judgement, over the pairs it answered (refusals and malformed outputs left out); temperature 0 (claude-opus-4-7 at the provider default), 1,024 output tokens, on 250 TruthfulQA statements judged accurate or inaccurate; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 25 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemma 4 31B (IT)0.98#350
2Gemini 3.7 Flash0.976#41
3Claude Opus 4.70.96#85
4Qwen 3.7 Flash (2026-07-15)0.96
5Claude Haiku 4.5 (20251001)0.956#543
6Claude Sonnet 4.50.952#261
7Llama 3.3 70B Instruct0.952#1385
8Qwen 3 32B0.952#972
9Kimi K30.948#61
10Qwen 3.6 35B A3B0.932#481
11Llama 4 Maverick Instruct FP80.916#1023
12Qwen 2.5 72B Instruct0.908#1117
13DeepSeek V4 Pro (0813)0.904#97
14Gemini 2.5 Flash0.896#463
15Llama 4 Scout Instruct0.892#1579

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=judgesense-factuality · How It Works · Data refreshed daily, snapshot 2026-11-21.