NuclearBench - Adversarial FineSurE Faithfulness: leaderboard
Long-document nuclear regulatory and engineering question answering: models receive the full source document and are graded for rubric criteria coverage and sentence-level faithfulness. NuclearBench (Atomic Canyon with Idaho National Laboratory SMEs): 280 published questions over 247 public NRC and nuclear technical documents; the model receives the question and the complete source document (no retrieval). CriteriaJudge marks each rubric criterion met or not met without the document; criteria score is the mean per-question fraction met. FineSurE classifies each answer sentence against the document; the score is the mean share of error-free sentences. The 125-question holdout is not used. Per-model temperature, reasoning effort and provider pins are kept in the gated models.yaml; the public card names no judge model. Adversarial set (61 questions). Adversarial items plant errors, strip identifiers or pair a mismatched document; refusing or correcting the premise is acceptable. Question access: The question and document files on Hugging Face require accepting access terms (unauthenticated README is public; models.yaml and JSONL return 401). A 125-question holdout is withheld.
Metric: FineSurE faithfulness score (%). Source: huggingface.co. Saturation forecast: Rough model projection: around 2026. 16 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-6 Astra (Benchmark Sweep Run: source=NuclearBench; adversarial set; full source document in context; settings in gated models.yaml) | 91.2 | |
| 2 | Claude Opus 5.5 (Benchmark Sweep Run: source=NuclearBench; adversarial set; full source document in context; settings in gated models.yaml) | 88.5 | |
| 3 | GPT-6.1 Sol (Benchmark Sweep Run: source=NuclearBench; adversarial set; full source document in context; settings in gated models.yaml) | 88.2 | |
| 4 | DeepSeek V4 Pro (Benchmark Sweep Run: source=NuclearBench; adversarial set; full source document in context; settings in gated models.yaml) | 87.7 | |
| 5 | Claude Fable 5.1 (Benchmark Sweep Run: source=NuclearBench; adversarial set; full source document in context; settings in gated models.yaml) | 87.4 | |
| 6 | GPT-6.1 Terra (Benchmark Sweep Run: source=NuclearBench; adversarial set; full source document in context; settings in gated models.yaml) | 86.9 | |
| 7 | Gemini 3.8 Flash (Benchmark Sweep Run: source=NuclearBench; adversarial set; full source document in context; settings in gated models.yaml) | 86.3 | |
| 8 | Nemotron 3.5 Super VL (Benchmark Sweep Run: source=NuclearBench; adversarial set; full source document in context; settings in gated models.yaml) | 86 | |
| 9 | Claude Sonnet 5.5 (Benchmark Sweep Run: source=NuclearBench; adversarial set; full source document in context; settings in gated models.yaml) | 85.2 | |
| 10 | Claude Haiku 5.5 (Benchmark Sweep Run: source=NuclearBench; adversarial set; full source document in context; settings in gated models.yaml) | 85 |
Interactive version: theaggregate.ai/benchmark?slug=nuclearbench-adversarial-finesure-faithfulness · How It Works · Data refreshed daily, snapshot 2026-11-07.