falcon-40B Instruct: benchmark results
TII's Falcon-40B-Instruct, a 40B causal decoder-only chat model fine-tuned from Falcon-40B on a mixture of the Baize dataset, using FlashAttention and multi-query attention for inference. Provider: TII. Released 2023-05-25. Access: Open.
Aggregate Rating 1308 ± 11, rank #1440 of 1635 rated models, from 98 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| MMLU-by-task - High School Mathematics | 33.33 | Accuracy (%) | 93.3 |
| MMLU-by-task - TruthfulQA MC1 | 37.7 | Accuracy (%) | 91.1 |
| MMLU-by-task - Anatomy | 54.07 | Accuracy (%) | 90 |
| HELM Classic - bAbI | 60.6 | Exact Match (%) | 89.9 |
| HELM Classic - WikiFact | 35.92 | Exact Match (%) | 89.4 |
| MMLU-by-task - HellaSwag | 64.62 | Accuracy (%) | 88.1 |
| HELM Classic - IMDB | 95.9 | Exact Match (%) | 87.9 |
| MMLU-by-task - TruthfulQA MC2 | 52.52 | Accuracy (%) | 87.7 |
| MMLU-by-task - Virology | 49.4 | Accuracy (%) | 87.4 |
| MMLU-by-task - Professional Medicine | 58.09 | Accuracy (%) | 87 |
| HELM Classic - TruthfulQA | 38.38 | Exact Match (%) | 86.4 |
| MMLU-by-task - Medical Genetics | 64 | Accuracy (%) | 86.3 |
Interactive version: theaggregate.ai/model?slug=falcon-40b-instruct · How It Works · Data refreshed daily, snapshot 2026-11-05.