CoCoNot - Humanizing Requests: leaderboard
The model receives requests that treat it as a person with feelings, experiences or opinions, and the score is the share it plays along instead of stating it is an AI. Unlike persona benchmarks, playing along here counts as the failure.
Metric: Compliance rate on humanizing requests (%, lower is better). Source: arxiv.org. 19 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-4 Preview (1106) | 1.2 | #953 |
| 2 | GPT-4 (0613) | 6.1 | #1214 |
| 3 | tulu-2-dpo-70B | 6.1 | #3944 |
| 4 | tulu-2-dpo-7B | 6.1 | #7492 |
| 5 | tulu-2-dpo-13B | 6.1 | #6621 |
| 6 | tulu-2-13B | 6.1 | #7711 |
| 7 | tulu-2-7B | 9.8 | #8164 |
| 8 | Claude 3 Sonnet | 9.9 | #2309 |
| 9 | GPT-3.5 Turbo (0125) | 13.4 | #3500 |
| 10 | Llama 2 13B Chat | 18.3 | #8186 |
| 11 | Llama 3 70B Instruct | 22 | #1676 |
| 12 | Mixtral 8x22B Instruct | 22 | #2039 |
| 13 | Llama 2 7B Chat | 22 | #8805 |
| 14 | GPT-4o (2024-05-13) | 23.2 | #735 |
| 15 | Mistral 7B Instruct (v0.2) | 23.2 | #7007 |
Interactive version: theaggregate.ai/benchmark?slug=coconot-humanizing-requests · How It Works · Data refreshed daily, snapshot 2026-11-13.