AbstentionBench - underspecified intent - CoCoNot/Incomprehensible - Recall Leaderboard · The Aggregate

AbstentionBench - underspecified intent - CoCoNot/Incomprehensible - Recall: leaderboard

Metric: Abstention Recall (%). Source: github.com. 20 models tracked.

Top models

#	Model	Score
1	Qwen 2.5 32B Instruct	95.92
2	Llama-3.1-Tulu-3-8B-SFT	95.92
3	GPT-4o	93.88
4	Llama-3.1-Tulu-3-70B-DPO	93.88
5	Llama 3.1 8B Instruct	91.84
6	Llama-3.1-Tulu-3-8B	91.84
7	Gemini 1.5 Pro	89.8
8	Llama-3.1-Tulu-3-70B	89.8
9	Llama-3.1-Tulu-3-8B-DPO	87.76
10	Llama 3.1 405B Instruct	85.71
11	O1	85.71
12	Llama 3.1 70B Instruct	83.67
13	Mistral 7B Instruct (v0.3)	83.67
14	Llama 3.3 70B Instruct	71.43
15	DeepSeek R1 Distill Llama 70B	61.22

Interactive version: theaggregate.ai/benchmark?slug=abstentionbench-underspecified-intent-coconot-incomprehensible-recall · How It Works · Data refreshed daily, snapshot 2026-09-19.