ICE-Guard: leaderboard
The model makes yes/no-style decisions on up to 300 paired vignettes, drawn from a 3,000-vignette benchmark, in ten high-stakes domains (hiring, lending, triage, sentencing and others), each asked again with one decision-irrelevant detail swapped (a name or demographic marker, a credential, or the framing); the number is the share of pairs whose decision flips, averaged over the three swap types. Unlike BBQ, it scores the model's own decisions, not answers about stereotypes.
Metric: Decision flip rate under an irrelevant swap (%, lower is better). Source: arxiv.org. 10 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GLM-5 | 1.1 | #260 |
| 2 | GPT-OSS-120B | 1.3 | #700 |
| 3 | Claude Opus 4.6 | 1.3 | #102 |
| 4 | GPT-5.2 | 3.7 | #176 |
| 5 | Gemini 3 Flash | 4 | #164 |
| 6 | Gemini 2.5 Pro | 5.3 | #266 |
| 7 | DeepSeek V3.2 | 5.3 | #406 |
| 8 | Kimi K2.5 | 5.5 | #256 |
| 9 | Claude Sonnet 4.5 | 6 | #250 |
| 10 | Qwen 3 32B | 7.7 | #960 |
Interactive version: theaggregate.ai/benchmark?slug=ice-guard · How It Works · Data refreshed daily, snapshot 2026-11-13.