PluriHarms: leaderboard
PluriHarms (Li et al., 2026) tests how well a model predicts one person's harm ratings: the same 100 U.S. annotators rated 150 benign-to-harmful prompts from 0 to 100, and the model is shown that annotator's ratings from the 100-prompt alignment set in context (k-shot) before predicting the same person's ratings of the 50 held-out prompts. The score is mean absolute error on a 0-1 scale averaged over annotators, lower is better; the paper gives refusal and completion rates separately and does not say how unanswered predictions enter the error.
Metric: Mean Absolute Error vs Individual Ratings (0-1; k-shot personalised steering). Source: arxiv.org. Saturation forecast: Around February 2027. 13 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5 | 0.2 | #163 |
| 2 | GPT-4.1 | 0.2 | #492 |
| 3 | Qwen 3 8B | 0.2 | #1838 |
| 4 | Claude 3.7 Sonnet | 0.2 | #455 |
| 5 | Claude Opus 4 | 0.2 | #317 |
| 6 | Qwen 3 32B | 0.21 | #960 |
| 7 | Claude Sonnet 4 | 0.21 | #397 |
| 8 | Claude Sonnet 4.5 | 0.21 | #244 |
| 9 | Qwen 3 14B | 0.21 | #1235 |
| 10 | Claude 3.5 Haiku | 0.21 | #1563 |
| 11 | Claude Haiku 4.5 | 0.22 | #568 |
| 12 | Qwen 3 4B | 0.23 | #2297 |
| 13 | Claude 3 Haiku | 0.23 | #2629 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=pluriharms · How It Works · Data refreshed daily, snapshot 2026-11-12.