Llama-3.1-Tulu-3-8B-DPO: benchmark results
Ai2's DPO-stage checkpoint of the fully open Tulu 3 post-training recipe, built on Llama 3.1 8B (November 2024). Provider: Allen AI. Released 2024-11-21. Access: Open.
Aggregate Rating 1450 ± 23, rank #1000 of 1656 rated models, from 173 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open LLM Leaderboard - IFEval | 80.29 | Score | 97.7 |
| AbstentionBench - underspecified context - UMWP - Precision | 100 | Abstention Precision (%) | 97.2 |
| AbstentionBench - underspecified context - QASPER - F1 | 44.51 | Abstention F1 (%) | 94.7 |
| AbstentionBench - underspecified context - QASPER - Precision | 29.07 | Abstention Precision (%) | 94.7 |
| EuroEval Portuguese NLU - MultiWikiQA PT | 76.29 | Character F1 (%) | 90.1 |
| EuroEval Dutch NLU - DBRD | 91.49 | Matthews correlation coefficient (%) | 87.3 |
| AbstentionBench - answer unknown - KUQ/Future unknowns - Recall | 73.62 | Abstention Recall (%) | 84.2 |
| AbstentionBench - stale - FreshQA - F1 | 48.78 | Abstention F1 (%) | 84.2 |
| AbstentionBench - subjective - CoCoNot/Subjective - F1 | 92.86 | Abstention F1 (%) | 84.2 |
| AbstentionBench - subjective - CoCoNot/Subjective - Recall | 86.67 | Abstention Recall (%) | 84.2 |
| AbstentionBench - underspecified intent - SituatedQA/Geo - Recall | 79.07 | Abstention Recall (%) | 84.2 |
| Enkrypt AI - Toxicity Risk | 0.95 | Risk Score | 83.8 |
Interactive version: theaggregate.ai/model?slug=llama-3-1-tulu-3-8b-dpo · How It Works · Data refreshed daily, snapshot 2026-10-20.