Loading data...
Falcon-H1-1.5B-Deep-Base: benchmark results
Provider: TII. Released 2025-05-20. Access: Open.
Aggregate Rating 1412 ± 33, rank #5677 of 10552 rated models, from 37 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
| Lara Mini Base Report (base-model protocol) - XNLI (English) (0-shot) | 55.9 | Accuracy (%), English benchmark | 94.4 |
| Lara Mini Base Report (base-model protocol) - MBPP (3-shot) | 56.8 | pass@1 (execution) (%) | 88.9 |
| Lara Mini Base Report (base-model protocol) - MMLU-Pro (5 (CoT)-shot) | 40.1 | Exact match (%), English benchmark | 88.9 |
| Lara Mini Base Report (base-model protocol) - MMLU (5-shot) | 66.6 | Accuracy (%), English benchmark | 87 |
| Lara Mini Base Report (base-model protocol) - GSM8K (5-shot) | 72.3 | Exact match (strict extraction: first "#### N", else first \ | 85.2 |
| Lara Mini Base Report (base-model protocol) - PAWS (English) (0-shot) | 68 | Accuracy (%), English benchmark | 85.2 |
| Lara Mini Base Report (base-model protocol) - MBPP+ (3-shot) | 58.5 | pass@1; must pass MBPP's own and the EvalPlus tests (%) | 84.3 |
| Lara Mini Base Report (base-model protocol) - MATH (4-shot) | 49.2 | Math-Verify equivalence, Minerva prompts, 5,000 problems ove | 83.3 |
| Lara Mini Base Report (base-model protocol) - CRUXEval-O (2-shot) | 37.8 | pass@1, direct prediction, official answer parser (%) | 82.4 |
| Lara Mini Base Report (base-model protocol) - MGSM (8 (CoT)-shot) | 44.3 | Exact match on each language's native CoT answer phrase, ave | 74.1 |
| Lara Mini Base Report (base-model protocol) - CRUXEval-I (2-shot) | 35.8 | pass@1, direct prediction, official answer parser (%) | 72.2 |
| Lara Mini Base Report (base-model protocol) - PAWS-X, All Languages (0-shot) | 56.8 | Accuracy (%) averaged over all languages/variants of the ben | 68.5 |
Interactive version: theaggregate.ai/model?slug=falcon-h1-1-5b-deep-base · How It Works · Data refreshed daily, snapshot 2026-11-16.