IndicIPR-QA - Zero-Shot F1: leaderboard
IndicIPR-QA (arXiv 2610.40518): extractive question answering over Indian intellectual-property statutes (patents, trademarks, copyright, designs), 344 test questions whose answer is a span of the statute text. Token-overlap F1 of the extracted span, zero-shot, for open models run as Ollama default builds.
Metric: Token F1 (%), zero-shot. Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 3 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | qwen2.5:7b (Ollama) | 77.59 | |
| 2 | llama3.1:8b (Ollama) | 74.78 | |
| 3 | mistral:7b (Ollama) | 67.11 |
Interactive version: theaggregate.ai/benchmark?slug=indicipr-qa-zero-shot-f1 · How It Works · Data refreshed daily, snapshot 2026-11-05.