BDBBQ Certified - Native Chat - ben_acc_ambig: leaderboard
BDBBQ official certified2026-10-30JSON,30656questions perlanguage,61312permodel,English/Bengali andoverall/ambiguous/disambiguated cohorts explicit. BF16,L40S,vLLM,context2048,temp0. Seven nativechatmodels16tokens;BanglaLLM ChatML is separate from paperplaincompletion5tokens and is not substituted. Ambiguous correctunknown;disambiguated factualgold,invalidoutputwrong,answerpermutationsseparaterecords,nomajorityvote. Accuracy only; signedbias and parsefailrates are diagnostics.
Metric: Score (%). Source: raw.githubusercontent.com. Saturation forecast: Estimated already saturated. 7 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Mistral Nemo Instruct (2407) | 79.35 | #2978 |
| 2 | Gemma 2 9B (IT) | 77.7 | #2275 |
| 3 | Qwen 2.5 7B Instruct | 47.07 | #2583 |
| 4 | Ministral-8B-Instruct-2410 | 43.22 | #4720 |
| 5 | Llama 3.1 8B Instruct | 38.88 | #4277 |
| 6 | Qwen 2.5 3B Instruct | 18.26 | #5206 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=bdbbq-certified-native-chat-ben-acc-ambig · How It Works · Data refreshed daily, snapshot 2026-11-12.