LookZoom 8B 98 Tokens - Harm: leaderboard
LookZoom v1: Qwen3-VL-Instruct at 2B/4B/8B. Fixed 500-question subsets (seed 0) of DocVQA val (ANLS),InfographicVQA val (ANLS),ChartQA test (relaxed accuracy),TextVQA val (VQA accuracy), plus 300 held-out V-CoT-Doc (strict match). Per-view budget B means B * 32^2 pixels preserving aspect ratio, not total tokens. Single view/AttWarp/replace answer from 1 view; ViCrop/ground-then-crop/GapSight/LookZoom use 2. LookZoom reader uses GRPO LoRA; pointer trained from own correct answers; GapSight is authors reimplementation. Distinct trained systems remain separate. TableS11 pools 2300 questions per cell. Rescue is the fraction of this backbone single-view base errors corrected; harm is the fraction of base correct answers broken. Correct means score>=0.5, except V-CoT strict match. Backbones and budgets have distinct baseline-conditioned denominators, so no cross-backbone pooling. Metric: Harm (%).
Metric: Harm (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen3-VL-8B-Instruct (Harness: LookZoom study LookZoom) | 6.7 | |
| 2 | Qwen3-VL-8B-Instruct (Harness: LookZoom study GapSight reimplementation) | 7.6 | |
| 3 | Qwen3-VL-8B-Instruct (Harness: LookZoom study Ground-then-crop) | 7.7 | |
| 4 | Qwen3-VL-8B-Instruct (Harness: LookZoom study ViCrop) | 8.7 | |
| 5 | Qwen3-VL-8B-Instruct (Harness: LookZoom study LookZoom replace) | 9.8 | |
| 6 | Qwen3-VL-8B-Instruct (Harness: LookZoom study AttWarp) | 20.9 |
Interactive version: theaggregate.ai/benchmark?slug=lookzoom-8b-98-tokens-harm · How It Works · Data refreshed daily, snapshot 2026-10-27.