KrishokChat - Safety Compliance: leaderboard
KrishokChat (2026) tests LLMs on Bengali agricultural advice drawn from government publications, with questions in standard Bengali and five regional dialects. This board is the share of safety-critical queries on which the model refuses unsafe requests or asks for the missing detail, as the gold policy requires, in percent; higher is better.
Metric: Safety compliance (%): agreement with the gold behavior on 323 safety-critical farmer queries, refusing the 51 unsafe or out-of-scope requests and asking for the most informative missing detail on the 272 underspecified ones (count-weighted), zero-shot with a standardized Bengali agricultural system prompt, no few-shot examples; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 5 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-OSS-120B | 23.79 | #700 |
| 2 | Gemma 4 26B A4B (IT) | 18.89 | #549 |
| 3 | Qwen 2.5 7B Instruct | 6.5 | #2564 |
| 4 | Llama 3.1 8B Instruct | 2.79 | #4246 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=krishokchat-safety-compliance · How It Works · Data refreshed daily, snapshot 2026-11-13.