CAT — MASK factual accuracy — SFT-tu: leaderboard
Table 3 accuracy column, method=SFT-tu, Gemma-4-31B-it judge. This is factual accuracy, distinct from honesty/belief consistency and from dishonesty rate. Qwen reasoning OFF. Unmodified checkpoint for Base; each trained adapter remains a source-qualified identity. Undetermined answers remain in the published denominator; no conversion to conditional accuracy. Metric: Accuracy (%). Higher is better. Source: https://arxiv.org/html/2610.22292v1
Metric: Accuracy (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 2 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Llama 3.3 70B Instruct (SFT-tu, CAT 2610.22292) | 78.4 | |
| 2 | Qwen3.6 27B (Non-reasoning) (SFT-tu, CAT 2610.22292) | 70.1 |
Interactive version: theaggregate.ai/benchmark?slug=cat-mask-factual-accuracy-sft-tu · How It Works · Data refreshed daily, snapshot 2026-10-17.