Falcon-H1-1.5B-Deep-Base: benchmark results

Provider: TII. Released 2025-05-20. Access: Open.

Aggregate Rating 1412 ± 33, rank #5677 of 10552 rated models, from 37 benchmark results.

Strongest benchmark results

BenchmarkScoreMetricPercentile
Lara Mini Base Report (base-model protocol) - XNLI (English) (0-shot)55.9Accuracy (%), English benchmark94.4
Lara Mini Base Report (base-model protocol) - MBPP (3-shot)56.8pass@1 (execution) (%)88.9
Lara Mini Base Report (base-model protocol) - MMLU-Pro (5 (CoT)-shot)40.1Exact match (%), English benchmark88.9
Lara Mini Base Report (base-model protocol) - MMLU (5-shot)66.6Accuracy (%), English benchmark87
Lara Mini Base Report (base-model protocol) - GSM8K (5-shot)72.3Exact match (strict extraction: first "#### N", else first \85.2
Lara Mini Base Report (base-model protocol) - PAWS (English) (0-shot)68Accuracy (%), English benchmark85.2
Lara Mini Base Report (base-model protocol) - MBPP+ (3-shot)58.5pass@1; must pass MBPP's own and the EvalPlus tests (%)84.3
Lara Mini Base Report (base-model protocol) - MATH (4-shot)49.2Math-Verify equivalence, Minerva prompts, 5,000 problems ove83.3
Lara Mini Base Report (base-model protocol) - CRUXEval-O (2-shot)37.8pass@1, direct prediction, official answer parser (%)82.4
Lara Mini Base Report (base-model protocol) - MGSM (8 (CoT)-shot)44.3Exact match on each language's native CoT answer phrase, ave74.1
Lara Mini Base Report (base-model protocol) - CRUXEval-I (2-shot)35.8pass@1, direct prediction, official answer parser (%)72.2
Lara Mini Base Report (base-model protocol) - PAWS-X, All Languages (0-shot)56.8Accuracy (%) averaged over all languages/variants of the ben68.5

Interactive version: theaggregate.ai/model?slug=falcon-h1-1-5b-deep-base · How It Works · Data refreshed daily, snapshot 2026-11-16.