TinyLlama-1.1B-Chat-v1.0: benchmark results
The TinyLlama project's 1.1B chat model, sharing Llama 2's architecture and tokenizer, fine-tuned from a 3-trillion-token checkpoint using Hugging Face's Zephyr recipe of UltraChat SFT plus UltraFeedback DPO. Provider: Singapore Univ. Released 2023-12-30. Access: Open.
Aggregate Rating 1221 ± 37, rank #1567 of 1654 rated models, from 47 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| AbstentionBench - underspecified context - MMLU History - F1 | 34.62 | Abstention F1 (%) | 40 |
| AbstentionBench - underspecified context - MMLU History - Recall | 23.08 | Abstention Recall (%) | 40 |
| Open PL LLM - PSC (multiple choice, 5-shot) | 40.5 | Binary F1 (%) | 27 |
| Open PL LLM - EQ-Bench | 2.64 | EQ-Bench Score | 20.3 |
| Open PL LLM - CBD (multiple choice, 5-shot) | 15.46 | Macro F1 (%) | 18.4 |
| Open PL LLM - DYK (multiple choice, 5-shot) | 24.85 | Binary F1 (%) | 17.8 |
| Open PL LLM - Multiple Choice | -5.32 | Average Multiple-Choice Score (%) | 17.8 |
| Open PL LLM - PolQA Closed Book (generative, 5-shot) | 18.9 | Levenshtein Similarity (%) | 17.6 |
| Open Portuguese LLM - FaQuAD NLI | 43.97 | Macro F1 (%) | 17.1 |
| Open PL LLM - PPC (multiple choice, 5-shot) | 42.9 | Accuracy (%) | 16.4 |
| Open PL LLM - RAG | 20.8 | Average RAG Score (%) | 16.4 |
| AbstentionBench - underspecified context - MMLU Math - Precision | 88.89 | Abstention Precision (%) | 15.8 |
Interactive version: theaggregate.ai/model?slug=tinyllama-1-1b-chat-v1-0 · How It Works · Data refreshed daily, snapshot 2026-10-24.