Llama 3.1 8B Fireplace2: benchmark results
ValiantLabs' chat fine-tune of Llama 3.1 8B Instruct that adds structured outputs, including inline function calls, SQL queries, JSON objects and matplotlib visualizations, within normal chat. Provider: Valiant Labs. Released 2024-07-23. Access: Open.
Aggregate Rating 1364 ± 21, rank #1346 of 1654 rated models, from 20 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Open Korean LLM - Ko-GPQA-Diamond | 27.27 | Normalized accuracy (%) | 92.4 |
| Open LLM Leaderboard - IFEval | 54.83 | Score | 66.2 |
| Open Korean LLM - Ko-GSM8K (flexible extract) | 28.28 | Exact match, flexible extract (%) | 62.7 |
| Open Korean LLM - Ko-IFEval | 31.91 | Strict accuracy, prompt/instruction mean (%) | 61.1 |
| Open Korean LLM Leaderboard | 36.48 | Average Score (%) | 57.3 |
| Open Korean LLM - Ko-EQ-Bench | 8.85 | EQ-Bench score | 55 |
| Open Portuguese LLM - OAB Exams | 42.82 | Accuracy (%) | 53.4 |
| Open Korean LLM - KorNAT-Helpful | 47 | Normalized accuracy (%) | 51.7 |
| Open LLM Leaderboard - GPQA | 28.94 | Score | 44 |
| Open LLM Leaderboard - MATH Level 5 | 8.76 | Score | 43.4 |
| Open Portuguese LLM - ENEM | 56.75 | Accuracy (%) | 40.8 |
| Open Portuguese LLM - BLUEX | 47.84 | Accuracy (%) | 40.2 |
Interactive version: theaggregate.ai/model?slug=llama-3-1-8b-fireplace2 · How It Works · Data refreshed daily, snapshot 2026-10-24.