Molmo2-4B: benchmark results
Provider: Allen AI. Released 2025-12-11. Access: Open.
Aggregate Rating 1572 ± 23, rank #1366 of 10916 rated models, from 59 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| VLM-RobustBench (MMMU-Pro) - Severe-Failure Rate | 3.8 | Severe-failure rate (%): share of the 133 corrupted settings | 93.8 |
| VSTAT - Atomic | 37.1 | Score (%) on questions about atomic state structures: 1,500 | 90.9 |
| BURST-VideoCount | 61.5 | Exact-count accuracy (%; 2.2k queries from BURST test-set tr | 90 |
| VSTAT - Location | 39.7 | Score (%) on questions about location-type state elements: 1 | 86.4 |
| Molmo2-VideoCount | 34.3 | Exact-count accuracy (%; 533 validation queries) | 80 |
| VSTAT | 34.4 | Average score (%) over all questions: 1,500 questions on 834 | 77.3 |
| VSTAT - Count | 31.6 | Score (%) on questions about count-type state elements: 1,50 | 77.3 |
| Molmo2-CapTest | 39.9 | Caption F1 (%; LLM-judged statement precision and recall aga | 70 |
| VLM-RobustBench (MMBench) - Severe-Failure Rate | 4.5 | Severe-failure rate (%): share of the 133 corrupted settings | 68.8 |
| Molmo2 Human Preference | 1041 | Elo rating (bootstrap median, Bradley-Terry; pairwise human | 65 |
| PhysCount - Image Formation NAE | 34.68 | Image-formation normalized absolute error (%) | 64.1 |
| VLM-RobustBench (MMMU-Pro) | 4.9 | Mean relative corruption error (percent of visual gain lost) | 62.5 |
Interactive version: theaggregate.ai/model?slug=molmo2-4b · How It Works · Data refreshed daily, snapshot 2026-11-08.