Mantis-8B-Idefics2: benchmark results
Provider: Salesforce. Released 2024-05-29. Access: Open.
Aggregate Rating 1351 ± 37, rank #7673 of 10916 rated models, from 443 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| OpenVLM MMT-Bench - Point Tracking | 95 | Score (%) | 99.8 |
| OpenVLM MMT-Bench - MIA | 87.5 | Score (%) | 99.3 |
| OpenVLM MMT-Bench - Spot The Diff | 95 | Score (%) | 97.3 |
| OpenVLM A-Bench - Contradiction Overcome | 91.7 | Accuracy (%) | 96.9 |
| OpenVLM MMT-Bench - Geometrical Perspective | 91.7 | Score (%) | 96.6 |
| OpenVLM MMT-Bench - Human Object Interaction Recognition | 90 | Score (%) | 95.6 |
| OpenVLM MMT-Bench - VI | 66.6 | Score (%) | 95.1 |
| OpenVLM MMT-Bench - Pixel Recognition | 75 | Score (%) | 94.7 |
| OpenVLM MMT-Bench - Table Structure Recognition | 100 | Score (%) | 94.7 |
| OpenVLM MMT-Bench - Color Constancy | 71.4 | Score (%) | 94.2 |
| OpenVLM MMT-Bench - Sign Language Recognition | 45 | Score (%) | 91.7 |
| OpenVLM Q-Bench - Yes-or-No (Distortion) | 87.2 | Accuracy (%) | 90.8 |
Interactive version: theaggregate.ai/model?slug=mantis-8b-idefics2 · How It Works · Data refreshed daily, snapshot 2026-11-08.