h2o-danube3-500m-chat: benchmark results
H2O.ai's h2o-danube3-500m-chat, a 500M chat model using an adjusted Llama 2 architecture with the Mistral tokenizer, trained for an 8,192-token context and designed to run offline on phones. Provider: H2O.ai. Released 2024-07-04. Access: Open.
Aggregate Rating 1190 ± 39, rank #1594 of 1656 rated models, from 22 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Enkrypt AI - Jailbreak Risk | 11.84 | Risk Score | 34.2 |
| SLM Trustworthiness - Fairness | 43.27 | TrustLLM fairness accuracy (%; sub-task mean) | 30 |
| SLM Trustworthiness - Machine Ethics | 61.92 | TrustLLM machine-ethics accuracy (%; sub-task mean) | 20 |
| Open Portuguese LLM - FaQuAD NLI | 43.97 | Macro F1 (%) | 17.1 |
| Open LLM Leaderboard - IFEval | 22.08 | Score | 15.1 |
| Open LLM Leaderboard - MATH Level 5 | 1.66 | Score | 12.1 |
| Open Portuguese LLM - ENEM | 19.8 | Accuracy (%) | 11.5 |
| Open LLM Leaderboard - MuSR | 34.34 | Score | 10 |
| SLM Trustworthiness - Privacy | 29.02 | TrustLLM privacy accuracy (%; sub-task mean) | 10 |
| SLM Trustworthiness - Robustness | 56.85 | TrustLLM robustness accuracy (%; sub-task mean) | 10 |
| SLM Trustworthiness - TrustLLM Overall | 47.76 | TrustLLM score (%; mean of ethics, privacy, robustness and f | 10 |
| Open Portuguese LLM - OAB Exams | 23.37 | Accuracy (%) | 9.8 |
Interactive version: theaggregate.ai/model?slug=h2o-danube3-500m-chat · How It Works · Data refreshed daily, snapshot 2026-10-20.