JobCannon Protocol Sensitivity: leaderboard
A model takes four self-report questionnaires (Big Five, RIASEC, Dark Triad, Multiple Intelligences) under 13 protocol variants such as reversed options, a paraphrased instruction or the whole test in one prompt; the number is the median, over its 22 scales, of how far its scale score moves across variants, in human standard deviations. Unlike Stick To Your Role, it measures the self-report's stability under administration changes, not across simulated personas.
Metric: Median protocol range in human SD (lower is better): for each questionnaire scale, the spread (max minus min) of the model's score over its valid protocol variants, in units of the human test-taker standard deviation, then the median over the model's scales (table F11). Source: huggingface.co. 16 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-6 Astra | 0.51 | #6 |
| 2 | Claude Opus 5.5 | 0.6 | #5 |
| 3 | Claude Fable 5.1 | 0.75 | #15 |
| 4 | Claude Haiku 4.5 | 0.97 | #568 |
| 5 | Nova Pro | 1.06 | #1555 |
| 6 | Gemini 2.5 Pro | 1.18 | #266 |
| 7 | Gemma 3 27B (IT) | 1.19 | #1289 |
| 8 | Kimi K2.5 | 1.27 | #256 |
| 9 | Llama 4 Maverick | 1.34 | #1099 |
| 10 | GLM-5 | 1.41 | #260 |
| 11 | Qwen 3 VL 235B A22B | 1.47 | #479 |
| 12 | Mistral Large 3 | 1.59 | #817 |
| 13 | DeepSeek R1 | 1.75 | #473 |
| 14 | Qwen 3 32B | 1.79 | #960 |
| 15 | GPT-OSS-120B | 1.83 | #700 |
Interactive version: theaggregate.ai/benchmark?slug=jobcannon-protocol-sensitivity · How It Works · Data refreshed daily, snapshot 2026-11-13.