Ultralytics VLM Leaderboard - mAP50-95: leaderboard

The Ultralytics VLM Leaderboard (suite ul37-v2) ranks vision language models and open-vocabulary detectors on drawing boxes around named objects, the task behind auto-annotation. Unlike COCO or RefCOCO, which use everyday photographs, its 360 images come 10 each from 36 real-world detection datasets on Ultralytics Platform, among them dental X-rays, retinal lesions, retail products and baggage scans, sent whole with each dataset's own class names. Every image goes through the platform's production annotation call, and SAM 3.1 tightens every model's boxes except SAM 3's. A timeout, refusal or invalid output counts as an image with no predictions. Boxes are scored with the Ultralytics package's detection metrics in the order the model listed them, and the datasets are averaged equally. Providers run at their default reasoning effort unless that misses the platform's 115-second request limit. This board is mAP50-95: average precision averaged over IoU thresholds from 0.50 to 0.95, which rewards tighter boxes.

Metric: mAP50-95 over 36 UL37 detection datasets (0-1). Source: www.ultralytics.com. Saturation forecast: Around 2031. 44 models tracked.

Top models

Interactive version: theaggregate.ai/benchmark?slug=ultralytics-vlm-leaderboard-map50-95 · How It Works · Data refreshed daily, snapshot 2026-11-12.