IndicIPR-QA - 3-Shot EM: leaderboard
IndicIPR-QA (arXiv 2610.40518): extractive question answering over Indian intellectual-property statutes (patents, trademarks, copyright, designs), 344 test questions whose answer is a span of the statute text. Exact match of the extracted span with three worked examples in the prompt, for open models run as Ollama default builds.
Metric: Exact match (%), 3-shot. Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 3 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | qwen2.5:7b (Ollama) | 34.3 | |
| 2 | llama3.1:8b (Ollama) | 33.43 | |
| 3 | mistral:7b (Ollama) | 24.71 |
Interactive version: theaggregate.ai/benchmark?slug=indicipr-qa-3-shot-em · How It Works · Data refreshed daily, snapshot 2026-11-05.