MASK (Paper): leaderboard
The model's belief on a proposition is first elicited with neutral questions; it is then given a prompt that pressures it to say otherwise, and the number is the share of 1,500 examples where its statement contradicts its own belief. Unlike TruthfulQA-style accuracy boards, a model that is wrong but sincere does not lose points; this is the paper's own run, not the later Scale leaderboard (SEAL - MASK).
Metric: Examples where the model's statement under pressure contradicts its own elicited belief, P(lie) (%). Source: arxiv.org. 32 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Llama 3.2 1B Instruct | 13.1 | #9047 |
| 2 | Qwen 2.5 0.5B Instruct | 15.4 | #8987 |
| 3 | Llama 2 7B Chat | 21.5 | #8805 |
| 4 | Llama 3.1 8B Instruct | 23.5 | #4246 |
| 5 | Llama 3.2 3B Instruct | 23.5 | #7147 |
| 6 | Llama 2 13B Chat | 24.7 | #8186 |
| 7 | QwQ 32B-Preview | 25.2 | #1625 |
| 8 | Claude 3.7 Sonnet (20250219) | 26.6 | #402 |
| 9 | Llama 2 70B Chat | 26.7 | #6800 |
| 10 | Qwen 2.5 1.5B Instruct | 26.9 | #7470 |
| 11 | Llama 3.1 405B Instruct | 28 | #1115 |
| 12 | Claude 3.5 Sonnet (20240620) | 33.4 | #687 |
| 13 | Qwen 2.5 3B Instruct | 33.8 | #5150 |
| 14 | Qwen 1.5 7B Chat | 35.1 | #6361 |
| 15 | Qwen 1.5 110B Chat | 35.6 | #1771 |
Interactive version: theaggregate.ai/benchmark?slug=mask-paper · How It Works · Data refreshed daily, snapshot 2026-11-13.