BDBBQ Certified - Native Chat - eng_acc_disambig: leaderboard
BDBBQ official certified2026-10-30JSON,30656questions perlanguage,61312permodel,English/Bengali andoverall/ambiguous/disambiguated cohorts explicit. BF16,L40S,vLLM,context2048,temp0. Seven nativechatmodels16tokens;BanglaLLM ChatML is separate from paperplaincompletion5tokens and is not substituted. Ambiguous correctunknown;disambiguated factualgold,invalidoutputwrong,answerpermutationsseparaterecords,nomajorityvote. Accuracy only; signedbias and parsefailrates are diagnostics.
Metric: Score (%). Source: raw.githubusercontent.com. Saturation forecast: Estimated already saturated. 7 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Ministral-8B-Instruct-2410 | 98.66 | #4720 |
| 2 | Mistral Nemo Instruct (2407) | 98 | #2978 |
| 3 | Gemma 2 9B (IT) | 95.94 | #2275 |
| 4 | Qwen 2.5 7B Instruct | 95.69 | #2583 |
| 5 | Llama 3.1 8B Instruct | 94.85 | #4277 |
| 6 | Qwen 2.5 3B Instruct | 94.52 | #5206 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=bdbbq-certified-native-chat-eng-acc-disambig · How It Works · Data refreshed daily, snapshot 2026-11-12.