StylisticBias - Gender Disparity: leaderboard
StylisticBias (Kolli et al., 2026) asks multimodal LLMs 25 forced-choice social judgments (competent or incompetent, trustworthy or untrustworthy, wealthy or poor and others) about 500 generated studio portraits, in four option orders with three seeds each. Variation strength measures how much the rate of favourable choices differs across male and female faces, from 0 to 0.5 averaged over scenarios; lower is better.
Metric: Variation strength (0-0.5; between-group dispersion of the favourable-choice rate across male and female faces, averaged over 25 binary social-judgment scenarios on the unmodified base portraits; lower means judgments depend less on gender). Source: arxiv.org. Saturation forecast: Estimated already saturated. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3 VL 8B Instruct | 0.02 | #982 |
| 2 | InternVL3-14B | 0.02 | #1184 |
| 3 | Pixtral-12B | 0.03 | #2493 |
| 4 | gemma-4-E4B-it | 0.03 | #1462 |
| 5 | Gemma 3 12B (IT) | 0.04 | #1849 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=stylisticbias-gender-disparity · How It Works · Data refreshed daily, snapshot 2026-11-12.