IslamicLegalBench (Few-Shot) - Hallucination Rate: leaderboard
Metric: Hallucination rate (%): share of answers that are incorrect or partly correct with fabricated or misattributed content (refusals not counted), the unweighted mean of the 13 task rates, on IslamicLegalBench's 718 expert-built Islamic-law questions in 13 task types over classical fiqh texts of several schools (metadata and concept recall, source attribution, condition enumeration, statute synthesis, 'illah identification, analogy and maxim mapping), answers graded against the reference for every core element by an o3 judge (correct, partially correct, incorrect or refusal), temperature 0, few-shot prompting with format and reasoning exemplars; lower is better. Source: arxiv.org. Saturation forecast: Around January 2027. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4.5 | 19.18 | #138 |
| 2 | GPT-5 | 21.25 | #91 |
| 3 | Gemini 2.5 Pro | 27.53 | #145 |
| 4 | Grok 4 | 27.61 | #169 |
| 5 | Llama 4 Maverick Instruct FP8 | 31.72 | #403 |
| 6 | DeepSeek R1 | 33.75 | #245 |
| 7 | Qwen 3 235B A22B 2507 Instruct | 38.1 | #291 |
| 8 | Llama 3.1 8B Instruct | 57.44 | #1018 |
| 9 | GPT-OSS-120B | 59.21 | #330 |
No result here: #3 Claude Opus 5.5, #5 GPT-6 Astra, #8 Claude Fable 5.1.
Interactive version: theaggregate.ai/benchmark?slug=islamiclegalbench-few-shot-hallucination-rate · How It Works · Data refreshed daily, snapshot 2026-10-11.