Lumen-bench - Hindi: leaderboard
Lumen-bench harmful tool-action rate on the Hindi versions of the 108 scenarios (both system prompts, valid trials from 1,296 scheduled per model). It isolates conduct in one language where the headline pools all five.
Metric: Harmful tool-action rate (%, lower is better). Source: arxiv.org. 30 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) | 0 | #94 |
| 2 | GPT-5.4 | 0 | #123 |
| 3 | Claude Haiku 4.5 | 0 | #568 |
| 4 | DeepSeek V3.2 | 0 | #406 |
| 5 | Gemma 4 31B (IT) | 0 | #342 |
| 6 | MiniMax-M2.7 | 0 | #518 |
| 7 | Qwen 3.5 397B A17B | 0.08 | #246 |
| 8 | Qwen 3.5 27B | 0.08 | #359 |
| 9 | GPT-5 Mini (2025-08-07) | 0.08 | #343 |
| 10 | Gemini 3.1 Flash Lite (Preview) | 0.08 | #418 |
| 11 | Qwen 3.5 35B A3B | 0.31 | #482 |
| 12 | Gemma 4 26B A4B (IT) | 0.31 | #549 |
| 13 | GPT-OSS-120B | 0.54 | #700 |
| 14 | GPT-OSS-20B | 1.08 | #1147 |
| 15 | GPT-4.1 | 2.39 | #495 |
Interactive version: theaggregate.ai/benchmark?slug=lumen-bench-hindi · How It Works · Data refreshed daily, snapshot 2026-11-13.