Refusing the Impossible: leaderboard

Metric: Hallucination rate (%; share of the 270 unsatisfiable coding prompts, 24 subcategories in six languages, answered with ungrounded code instead of a refusal or an explicit flag; planted-entity items judged by a deterministic registry detector, the rest by a Claude Opus judge validated against human labels; Ollama at temperature 0). Source: arxiv.org. Saturation forecast: Around December 2026. 12 models tracked.

Top models

Interactive version: theaggregate.ai/benchmark?slug=refusing-the-impossible · How It Works · Data refreshed daily, snapshot 2026-09-26.