SAHM - Fatwa QA: leaderboard

Open-ended Arabic financial reasoning on SAHM (2026): the 2,000 open-ended Islamic finance fatwa questions, each answer scored from 0 to 10 by a Gemini 2.5 Flash judge against an expert-verified reference with an additive rubric. Higher is better.

Metric: Judge score (0-10) on the 2,000 open-ended Islamic finance fatwa questions of SAHM, an Arabic financial and Sharia-compliance benchmark, each answer graded against the gold answer by a Gemini 2.5 Flash judge with an additive rubric, greedy decoding, mean of three runs; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 20 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 3 Flash (Preview)9.16#152
2Claude Opus 4.58.81#160
3GPT-58.05#178
4Claude Sonnet 4.57.6#261
5Claude Haiku 4.56.53#583
6GPT-4o6.52#751
7Qwen 2.5 72B Instruct5.39#1117
8GPT-4o Mini5.31#1667
9Gemma 3 27B (IT)5.19#1324
10Llama 3.1 70B Instruct4.76#1571
11Fanar-1-9B-Instruct4.46#1844
12ALLaM-7B-Instruct-preview4.22#2001
13Qwen 2.5 14B Instruct4.06#1775
14Gemma 2 9B (IT)3.43#2369
15Qwen 2.5 7B Instruct2.68#2723

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=sahm-fatwa-qa · How It Works · Data refreshed daily, snapshot 2026-11-21.