IslamicLegalBench (Few-Shot) - Strict Accuracy: leaderboard
Metric: Correct response rate (%, no credit for partly correct answers): the unweighted mean of the 13 task rates, on IslamicLegalBench's 718 expert-built Islamic-law questions in 13 task types over classical fiqh texts of several schools (metadata and concept recall, source attribution, condition enumeration, statute synthesis, 'illah identification, analogy and maxim mapping), answers graded against the reference for every core element by an o3 judge (correct, partially correct, incorrect or refusal), temperature 0, few-shot prompting with format and reasoning exemplars; higher is better. Source: arxiv.org. Saturation forecast: Around January 2027. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5 | 67.65 | #91 |
| 2 | Claude Sonnet 4.5 | 65.63 | #138 |
| 3 | Gemini 2.5 Pro | 62.79 | #145 |
| 4 | Grok 4 | 61.69 | #169 |
| 5 | DeepSeek R1 | 54.21 | #245 |
| 6 | Qwen 3 235B A22B 2507 Instruct | 48.87 | #291 |
| 7 | Llama 4 Maverick Instruct FP8 | 46.78 | #403 |
| 8 | GPT-OSS-120B | 32.72 | #330 |
| 9 | Llama 3.1 8B Instruct | 30.96 | #1018 |
No result here: #3 Claude Opus 5.5, #5 GPT-6 Astra, #8 Claude Fable 5.1.
Interactive version: theaggregate.ai/benchmark?slug=islamiclegalbench-few-shot-strict-accuracy · How It Works · Data refreshed daily, snapshot 2026-10-11.