Bangladesh Legal QA (Supplied Law): leaderboard
This bilingual Bangladesh legal QA control set (North South University and BRAC University, 2026) tests whether small instruction-tuned models answer multiple-choice statutory questions, scored under strict consistency across all four option orders. Accuracy when the governing statutory section is supplied with four same-act distractors, in percent, higher is better.
Metric: Accuracy (%; the governing statutory section is supplied with four same-act distractors; over 150 supplied-law control MCQs, strict consistency (correct in all four option orders); reference instruction-tuned model). Source: arxiv.org. Saturation forecast: Estimated already saturated. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3.5 4B | 94 | #1180 |
| 2 | Gemma 4 E2B | 84.7 | #4124 |
| 3 | Qwen 3.5 2B | 76.7 | #4849 |
| 4 | Llama 3.2 3B | 68 | #7877 |
| 5 | Qwen 3.5 0.8B | 57.3 | #8139 |
| 6 | Llama 3.2 1B | 27.3 | #9314 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=bangladesh-legal-qa-supplied-law · How It Works · Data refreshed daily, snapshot 2026-11-12.