StylisticBias - Ethnicity Disparity: leaderboard
StylisticBias (Kolli et al., 2026) asks multimodal LLMs 25 forced-choice social judgments (competent or incompetent, trustworthy or untrustworthy, wealthy or poor and others) about 500 generated studio portraits, in four option orders with three seeds each. Variation strength measures how much the rate of favourable choices differs across Asian, African, European, Middle Eastern and Latino faces, from 0 to 0.5 averaged over scenarios; lower is better.
Metric: Variation strength (0-0.5; between-group dispersion of the favourable-choice rate across Asian, African, European, Middle Eastern and Latino faces, averaged over 25 binary social-judgment scenarios on the unmodified base portraits; lower means judgments depend less on ethnicity). Source: arxiv.org. Saturation forecast: Estimated already saturated. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3 VL 8B Instruct | 0.03 | #982 |
| 2 | InternVL3-14B | 0.03 | #1184 |
| 3 | gemma-4-E4B-it | 0.04 | #1462 |
| 4 | Pixtral-12B | 0.05 | #2493 |
| 5 | Gemma 3 12B (IT) | 0.05 | #1849 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=stylisticbias-ethnicity-disparity · How It Works · Data refreshed daily, snapshot 2026-11-12.