EgoIntrospect - Proactive Timing Judgment: leaderboard

Whether a proactive assistant may interrupt the wearer during a clip, judged on egocentric smart-glasses clips annotated by the wearers themselves (EgoIntrospect, 2026); native audio and video models hear the clip audio and the others read a transcript. Macro-F1 in percent over the interruptible and do-not-disturb classes on 269 clips; higher is better.

Metric: Macro-F1 (%) over the interruptible and do-not-disturb classes of a binary judgment of whether a proactive assistant may interrupt the wearer during a clip (269 clips, labels from the wearers' own do-not-disturb annotations, interruptible the majority class), fixation-overlaid egocentric clips from 15 held-out participants with the participant profile; native audio-video models hear the clip audio, the other models read its transcript; temperature 0 (0.6 for InternVL3.5 thinking mode); higher is better. Source: arxiv.org. Saturation forecast: Not forecast. 19 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Qwen 3 VL 8B Instruct60.69#1029
2MiniCPM-V-4.560#1441
3Qwen 3 Omni 30B A3B (Thinking)59.92
4Kimi K2.559.78#266
5InternVL2.5-8B59.06#2370
6Qwen 3.5 397B A17B58.36#264
7Qwen3 Omni 30B A3B Instruct57.62#930
8Qwen 3 VL 8B (Thinking)57.57#910
9Gemini 3 Flash (Preview)57.41#152
10Gemini 3.1 Pro (Preview)57.13#99
11LLaVA-OneVision-7B56.97#2981
12InternVL3-8B56.22#1782
13Qwen 3.5 Omni Plus53.22#254
14InternVL3.5-8B-Instruct52.78#1891
15LLaVA-Video-7B-Qwen239.6#1978

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=egointrospect-proactive-timing-judgment · How It Works · Data refreshed daily, snapshot 2026-11-21.