Gemini 1.5 Pro: benchmark results

Google's MoE-based Gemini 1.5 flagship, notable for long-context multimodal understanding of up to 2M tokens (February 2024). Provider: Google. Released 2024-02-15. Access: API.

Aggregate Rating 1599 ± 4, rank #445 of 1656 rated models, from 1077 benchmark results.

Strongest benchmark results

BenchmarkScoreMetricPercentile
AV-Odyssey - Music Score Matching29Score (%)100
AV-Odyssey - Speech Sentiment Analysis29.5Score (%)100
AVI-Bench - Audio-Visual Localization43.63Task score (%) on audio-visual localization (205 image and a100
AbstentionBench - answer unknown - KUQ/Unsolved problems - F182.54Abstention F1 (%)100
AbstentionBench - answer unknown - KUQ/Unsolved problems - Recall77.45Abstention Recall (%)100
AbstentionBench - false premise - FalseQA - F175.54Abstention F1 (%)100
AbstentionBench - false premise - FalseQA - Recall69Abstention Recall (%)100
AbstentionBench - subjective - KUQ/Controversial - F180.63Abstention F1 (%)100
AbstentionBench - subjective - KUQ/Controversial - Recall75.37Abstention Recall (%)100
AbstentionBench - underspecified context - MMLU History - F171.88Abstention F1 (%)100
AbstentionBench - underspecified context - MMLU History - Recall58.97Abstention Recall (%)100
AbstentionBench - underspecified context - QASPER - Recall98.73Abstention Recall (%)100

Interactive version: theaggregate.ai/model?slug=gemini-1-5-pro · How It Works · Data refreshed daily, snapshot 2026-10-20.