SAHM - Open-Ended QA Average: leaderboard

Arabic financial and Sharia-compliant reasoning on SAHM (2026): the mean of three open-ended tasks (event-cause reasoning over financial reports, AAOIFI Sharia standards questions and fatwa questions), each answer scored from 0 to 10 by a Gemini 2.5 Flash judge against expert-verified references. Higher is better.

Metric: Mean judge score (0-10) over the three open-ended tasks (event-cause QA, Islamic standards QA, fatwa QA) of SAHM, an Arabic financial and Sharia-compliance benchmark, each answer graded against the gold answer by a Gemini 2.5 Flash judge with an additive rubric, greedy decoding, mean of three runs; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 19 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Claude Opus 4.58.84#160
2GPT-58.84#178
3Claude Sonnet 4.58.4#261
4Claude Haiku 4.57.55#583
5GPT-4o7.16#751
6Gemma 3 27B (IT)6.69#1324
7Qwen 2.5 72B Instruct6.37#1117
8GPT-4o Mini6.29#1667
9Fanar-1-9B-Instruct5.67#1844
10Qwen 2.5 14B Instruct5.48#1775
11ALLaM-7B-Instruct-preview5.35#2001
12Llama 3.1 70B Instruct5.04#1571
13Gemma 2 9B (IT)4.93#2369
14Gemma 3 4B (IT)4.26#4139
15Qwen 2.5 7B Instruct4.06#2723

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=sahm-open-ended-qa-average · How It Works · Data refreshed daily, snapshot 2026-11-21.