IndicIPR-QA - 3-Shot F1: leaderboard
IndicIPR-QA (arXiv 2610.40518): extractive question answering over Indian intellectual-property statutes (patents, trademarks, copyright, designs), 344 test questions whose answer is a span of the statute text. Token-overlap F1 of the extracted span with three worked examples in the prompt, for open models run as Ollama default builds.
Metric: Token F1 (%), 3-shot. Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 3 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | llama3.1:8b (Ollama) | 77.02 | |
| 2 | qwen2.5:7b (Ollama) | 76.39 | |
| 3 | mistral:7b (Ollama) | 68.83 |
Interactive version: theaggregate.ai/benchmark?slug=indicipr-qa-3-shot-f1 · How It Works · Data refreshed daily, snapshot 2026-11-05.