SuperMemory-VQA (EgoButler) - Answerable F1: leaderboard
SuperMemory-VQA is an egocentric long-horizon memory question-answering benchmark of 4,853 QA pairs from 52.9 hours of Meta Aria glasses recordings across six memory tasks, with ordered answer choices (correct, vague, wrong, unanswerable). This board reports Binary answerable-vs-unanswerable F1 (%) under EgoButler; higher is better.
Metric: Binary answerable-vs-unanswerable F1 (%) under EgoButler; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 10 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.4 Mini | 75 | #389 |
| 2 | GPT-5.4 | 71.7 | #123 |
| 3 | Gemini 3 Flash | 71.2 | #164 |
| 4 | Gemma 4 31B (IT) | 43.9 | #342 |
| 5 | Gemini 3.1 Pro (Preview) | 43.5 | #94 |
| 6 | gemma-4-E4B-it | 30.9 | #1469 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=supermemory-vqa-egobutler-answerable-f1 · How It Works · Data refreshed daily, snapshot 2026-11-13.