Gemini 1.5 Pro (Sept): benchmark results
September 2024 Gemini 1.5 Pro snapshot, tracked when sources report the dated API model. Provider: Google. Released 2024-09-24. Access: API.
Aggregate Rating 1598 ± 21, rank #448 of 1656 rated models, from 20 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Deception Effectiveness (Lechmazur) | 0.94 | Deception Score | 76.5 |
| RewardBench Chat Hard | 76.97 | Accuracy (%) | 76.1 |
| RewardBench | 86.78 | Score (%) | 74.4 |
| LLM Public Goods Game | 24.64 | Avg. Contribution (%) | 73.7 |
| RewardBench Reasoning | 90.22 | Accuracy (%) | 73.4 |
| RewardBench Safety | 85.81 | Accuracy (%) | 63.1 |
| Confabulation Leaderboard (Lechmazur) | 16.83 | Confabulation rate % (lower is better) | 61.5 |
| RewardBench Chat | 94.13 | Accuracy (%) | 48.9 |
| AA MMLU-Pro | 75.02 | Accuracy (%) | 48.3 |
| Deception Resistance (Lechmazur) | 0.62 | Vulnerability Score (lower is better) | 47.1 |
| NYT Connections Original | 22.7 | Score (%) | 46.4 |
| AA LiveCodeBench | 31.64 | Pass@1 (%) | 37.4 |
Interactive version: theaggregate.ai/model?slug=gemini-1-5-pro-sept · How It Works · Data refreshed daily, snapshot 2026-10-20.