Social Caption - Directed Social Analysis: leaderboard
Metric: Human rating of a description of the social information needed to answer a Social-IQ 2.0 question about the video (sum of six 1-5 Likert sub-dimensions: relevant scene details, key individuals, relevant interactions, relevant context, answer detail, prompt adherence; 6-30; mean of two annotators per video; Social-IQ 2.0 validation videos (one minute each), a 50-video subset matched to the full set; the model gets the video at 1 frame per second and a timestamped WhisperX transcript, default temperature). Source: arxiv.org. Saturation forecast: Estimated already saturated. 10 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | Gemini 2.5 Pro | 25.94 |
| 2 | GPT-4o | 24.42 |
| 3 | MiniCPM-V-2.6 | 23.51 |
| 4 | Qwen 2.5 VL 7B | 23.49 |
| 5 | Gemini 1.5 Pro | 23.42 |
| 6 | InternVL3-8B | 23.01 |
| 7 | InternVL2-8B | 19.88 |
| 8 | Qwen 2 VL 7B | 18.9 |
| 9 | Qwen2.5-Omni-7B | 8.91 |
Interactive version: theaggregate.ai/benchmark?slug=social-caption-directed-social-analysis · How It Works · Data refreshed daily, snapshot 2026-09-26.