Social Caption - Holistic Social Analysis: leaderboard
A multimodal model watches a one-minute Social-IQ 2.0 video at one frame per second, with a WhisperX transcript, and writes a structured description of its social interaction: setting and mood, the individuals, topic and relationships, and emotional and power dynamics; Social Caption (Mathur et al., 2026). Two human annotators rate each description on six 1-5 sub-dimensions grounded in the APRACE taxonomy, and the score is the summed mean rating out of 30 over a 50-video subset of the validation set; higher is better.
Metric: Human rating of a holistic description of the video's social interaction (sum of six 1-5 Likert sub-dimensions: scene description, individuals, topic and context, socio-emotional analysis, answer detail, prompt adherence; 6-30; mean of two annotators per video; Social-IQ 2.0 validation videos (one minute each), a 50-video subset matched to the full set; the model gets the video at 1 frame per second and a timestamped WhisperX transcript, default temperature). Source: arxiv.org. Saturation forecast: Estimated already saturated. 10 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 2.5 Pro | 26.14 | #262 |
| 2 | Gemini 1.5 Pro | 24.46 | #1053 |
| 3 | Qwen 2.5 VL 7B | 23.24 | #1796 |
| 4 | InternVL2-8B | 23.13 | #3025 |
| 5 | GPT-4o | 22.85 | #741 |
| 6 | MiniCPM-V-2.6 | 22.35 | #3095 |
| 7 | InternVL3-8B | 21.78 | #1695 |
| 8 | Qwen 2 VL 7B | 21.63 | #2273 |
| 9 | Qwen2.5-Omni-7B | 14.84 | #1797 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=social-caption-holistic-social-analysis · How It Works · Data refreshed daily, snapshot 2026-11-12.