KrishokChat - Treatment QA (Oracle Context): leaderboard
KrishokChat (2026) tests LLMs on Bengali agricultural advice drawn from government publications, with questions in standard Bengali and five regional dialects. This board is the share of treatment questions answered with the exact correct chemical, dosage and unit when the gold source passage is provided, in percent; higher is better.
Metric: Treatment QA correct rate (%): the recommended treatment matches the provenance-tracked chemical, dosage and unit exactly (correct, omission or hallucination verdict against the chemical trace), 346 test questions in standard Bengali and five regional dialects across fertilizer, disease, pest, IPM, seed, food-safety and herbicide domains, zero-shot with a standardized Bengali agricultural system prompt, no few-shot examples, with the gold source passage in context; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 5 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemma 4 26B A4B (IT) | 54.05 | #549 |
| 2 | GPT-OSS-120B | 49.79 | #700 |
| 3 | Qwen 2.5 7B Instruct | 41.33 | #2564 |
| 4 | Llama 3.1 8B Instruct | 30.64 | #4246 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=krishokchat-treatment-qa-oracle-context · How It Works · Data refreshed daily, snapshot 2026-11-13.