Dealign DeepSeek1017 MMLU20x57 - Subject college biology: leaderboard
This publisher self-report evaluates a custom MMLU subset of 20 questions from each of 57 subjects (1140 total); exact sample identities are unpublished. It uses four-choice first-token argmax with an identical letter-only prompt, chat mode without reasoning, and greedy decoding at temperature 0. The base 1017 checkpoint is served by the official DeepSeek API; the derivative uses stock upstream inference on 4 x H200 GPUs with TP4. These different serving stacks remain explicit. This is neither full MMLU nor an independent replication. Scores are accuracy percentages. HarmBench refusal/compliance results and smoke tests are not treated as capability evidence. Each subject has 20 questions. Metric: Custom-subset first-token accuracy (%).
Metric: Custom-subset first-token accuracy (%). Source: huggingface.co. Saturation forecast: Rough model projection: around 2026. 2 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | DeepSeek-V4.1-Flash-1017 (no thinking) (Harness: dealign MMLU official API) | 90 | |
| 2 | DeepSeek-V4.1-Flash-1017-UNCENSORED-FP8 (no thinking) (Harness: dealign stock inference) | 90 |
Interactive version: theaggregate.ai/benchmark?slug=dealign-deepseek1017-mmlu20x57-subject-college-biology · How It Works · Data refreshed daily, snapshot 2026-10-28.