HaluCheck - English - HaluReal Span F1: leaderboard
Bilingual hallucination detection: factual perturbations in text (HaluReal), fabricated entities (HaluFake) and faithfulness hallucinations in agent trajectories (HaluAgent). HaluCheck v1 Lite evaluation sets (1,000 balanced HaluReal and 1,000 balanced HaluAgent samples per language; 500 HaluFake fabricated-entity samples per language); decoding temperature 0. Accuracy credits correct 1, uncertain 0.5, incorrect 0. SpanF1 matches predicted against gold hallucinated spans with an LLM matcher; CorrectionRate requires correctly localized spans to be corrected. RootTurnAcc identifies the earliest hallucinated agent turn. UncertainRate (abstention) and HaluFake recall on an all-fabricated set are not admitted: they measure disposition, and always flagging scores 100. Main-table thinking mode and reasoning effort are not stated (Figure 4 separately varies Qwen3 thinking); model rows keep their native checkpoints. Question access: Code repository github.com/xMiles/HaluCheck is cited; the exact Lite split release was not verified.
Metric: Hallucinated-span F1 (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 14 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.1 Pro (Benchmark Sweep Run: source=HaluCheck; Lite split; temperature=0; thinking/effort unreported) | 72.95 | |
| 2 | Gemini 3 Flash (Benchmark Sweep Run: source=HaluCheck; Lite split; temperature=0; thinking/effort unreported) | 60.12 | |
| 3 | DeepSeek V4 Pro (Benchmark Sweep Run: source=HaluCheck; Lite split; temperature=0; thinking/effort unreported) | 46.99 | |
| 4 | GPT-5.4 (Benchmark Sweep Run: source=HaluCheck; Lite split; temperature=0; thinking/effort unreported) | 42.88 | |
| 5 | GPT-5 Mini (Benchmark Sweep Run: source=HaluCheck; Lite split; temperature=0; thinking/effort unreported) | 42.61 | |
| 6 | DeepSeek V4 Flash (Benchmark Sweep Run: source=HaluCheck; Lite split; temperature=0; thinking/effort unreported) | 38.01 | |
| 7 | Hy3-preview (Benchmark Sweep Run: source=HaluCheck; Lite split; temperature=0; thinking/effort unreported) | 36.88 | |
| 8 | Qwen 3 235B A22B (Benchmark Sweep Run: source=HaluCheck; Lite split; temperature=0; thinking/effort unreported) | 28.33 | |
| 9 | Qwen 3 32B (Benchmark Sweep Run: source=HaluCheck; Lite split; temperature=0; thinking/effort unreported) | 26.63 | |
| 10 | Qwen 3 8B (Benchmark Sweep Run: source=HaluCheck; Lite split; temperature=0; thinking/effort unreported) | 25.38 |
Interactive version: theaggregate.ai/benchmark?slug=halucheck-english-halureal-span-f1 · How It Works · Data refreshed daily, snapshot 2026-11-07.