CAT — MASK factual accuracy — Honest persona + prefill: leaderboard
Table 3 accuracy column, method=Honest persona + prefill, Gemma-4-31B-it judge. This is factual accuracy, distinct from honesty/belief consistency and from dishonesty rate. Qwen reasoning OFF. Unmodified checkpoint for Base; each trained adapter remains a source-qualified identity. Undetermined answers remain in the published denominator; no conversion to conditional accuracy. Metric: Accuracy (%). Higher is better. Source: https://arxiv.org/html/2610.22292v1
Metric: Accuracy (%). Source: arxiv.org. Saturation forecast: Around January 2027. 2 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Llama 3.3 70B Instruct | 78.9 | #532 |
| 2 | Qwen 3.6 27B (Non-reasoning) | 46.4 | #167 (Qwen 3.6 27B) |
No result here: #3 Claude Opus 5.5, #5 GPT-6 Astra, #7 Gemini 4 Argon.
Interactive version: theaggregate.ai/benchmark?slug=cat-mask-factual-accuracy-honest-persona-plus-prefill · How It Works · Data refreshed daily, snapshot 2026-10-17.