Gemini 1.5 Pro (Sept) — benchmark results
September 2024 Gemini 1.5 Pro snapshot, tracked when sources report the dated API model. Provider: Google. Released 2024-09-24. Access: API.
Unified ELO 1552 ± 20, rank #616 of 1776 rated models, from 22 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| RewardBench | 86.78 | Score (%) | 86.1 |
| Deception Effectiveness (Lechmazur) | 0.94 | Deception Score | 76.5 |
| RewardBench Chat Hard | 76.97 | Accuracy (%) | 76.1 |
| LLM Public Goods Game | 24.64 | Avg. Contribution (%) | 73.7 |
| RewardBench Reasoning | 90.22 | Accuracy (%) | 73.4 |
| RewardBench Safety | 85.81 | Accuracy (%) | 66.7 |
| Confabulation Leaderboard (Lechmazur) | 16.83 | Confabulation rate % (lower is better) | 61.9 |
| AA MATH-500 | 87.6 | Accuracy (%) | 60.4 |
| RewardBench Chat | 94.13 | Accuracy (%) | 48.9 |
| AA MMLU-Pro | 75.02 | Accuracy (%) | 48.3 |
| Deception Resistance (Lechmazur) | 0.62 | Vulnerability Score (lower is better) | 47.1 |
| NYT Connections Original | 22.7 | Score (%) | 43.3 |
Interactive version: theaggregate.ai/model?slug=gemini-1-5-pro-sept · How the rankings work · Data refreshed daily, snapshot 2026-07-22.