Llama-3.1-Tulu-3-70B-DPO: benchmark results
Ai2's DPO-stage checkpoint of the fully open Tulu 3 post-training recipe, built on Llama 3.1 70B. Provider: Allen AI. Released 2024-11-21. Access: Open.
Aggregate Rating 1516 ± 30, rank #729 of 1656 rated models, from 103 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| AbstentionBench - answer unknown - KUQ/Future unknowns - Recall | 81.6 | Abstention Recall (%) | 100 |
| AbstentionBench - subjective - CoCoNot/Subjective - F1 | 97.26 | Abstention F1 (%) | 100 |
| AbstentionBench - subjective - CoCoNot/Subjective - Recall | 94.67 | Abstention Recall (%) | 100 |
| Open LLM Leaderboard - IFEval | 82.82 | Score | 99.2 |
| Open LLM Leaderboard - MuSR | 49.23 | Score | 97.3 |
| AbstentionBench - answer unknown - CoCoNot/Unsupported - F1 | 97.46 | Abstention F1 (%) | 94.7 |
| AbstentionBench - answer unknown - KUQ/Future unknowns - F1 | 85.26 | Abstention F1 (%) | 94.7 |
| AbstentionBench - subjective - KUQ/Controversial - F1 | 79.93 | Abstention F1 (%) | 94.7 |
| AbstentionBench - underspecified context - ALCUNA - F1 | 81.87 | Abstention F1 (%) | 94.7 |
| AbstentionBench - underspecified intent - SituatedQA/Geo - Recall | 88.37 | Abstention Recall (%) | 94.7 |
| Open LLM Leaderboard - GPQA | 37.58 | Score | 94 |
| Open LLM Leaderboard - MATH Level 5 | 44.94 | Score | 93.3 |
Interactive version: theaggregate.ai/model?slug=llama-3-1-tulu-3-70b-dpo · How It Works · Data refreshed daily, snapshot 2026-10-20.