Idefics3-8B: benchmark results
Provider: HuggingFace.
Aggregate Rating 1480 ± 31, rank #2662 of 10916 rated models, from 27 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| FewMMBench (8-Shot, Similar + CoT) | 70.1 | Mean accuracy (%) on the FewMMBench query sets (about 250 mu | 64 |
| FewMMBench (4-Shot, Similar + CoT) | 71 | Mean accuracy (%) on the FewMMBench query sets (about 250 mu | 60 |
| FewMMBench (8-Shot, Similar) | 72.4 | Mean accuracy (%) on the FewMMBench query sets (about 250 mu | 60 |
| FewMMBench (4-Shot, Similar) | 72.3 | Mean accuracy (%) on the FewMMBench query sets (about 250 mu | 56 |
| FewMMBench (8-Shot, Random) | 71.9 | Mean accuracy (%) on the FewMMBench query sets (about 250 mu | 54 |
| FewMMBench (4-Shot, Random) | 72.2 | Mean accuracy (%) on the FewMMBench query sets (about 250 mu | 52 |
| MedQ-Deg - Intensity Degradation | 65.7 | Accuracy (%) on MedQ-Deg items whose images carry intensity- | 51.3 |
| MedQ-Deg - Treatment Reasoning | 11.5 | Accuracy (%) on the Treatment questions of MedQ-Deg (capabil | 41 |
| FewMMBench (Zero-Shot) | 68.4 | Mean accuracy (%) on the FewMMBench query sets (about 250 mu | 40 |
| MedQ-Deg - Anatomical Recognition | 46.2 | Accuracy (%) on the Anatomical Recognition questions of MedQ | 38.5 |
| MedQ-Deg - Motion Degradation | 56.6 | Accuracy (%) on MedQ-Deg items whose images carry motion-int | 38.5 |
| MedQ-Deg - Diagnostic Reasoning | 25.2 | Accuracy (%) on the Diagnosis questions of MedQ-Deg (capabil | 37.2 |
Interactive version: theaggregate.ai/model?slug=idefics3-8b · How It Works · Data refreshed daily, snapshot 2026-11-08.