EpiQAL - Masked-Input Reasoning: leaderboard
EpiQAL-C (Emory University, University of Illinois Chicago and Microsoft, 2026): 479 single-answer questions asking which conclusion the authors of a PLOS Neglected Tropical Diseases article drew, with the Discussion section removed from the article the model reads, so the conclusion must be reconstructed from the remaining evidence. The score is exact match of the selected option set in percent, higher is better, zero-shot without chain-of-thought prompting, and selecting extra options fails the item. Qwen3-30B-A3B-Instruct-2507 generated the items, and those that a four-model difficulty pool found easy had their question stems rewritten.
Metric: Exact Match (%; zero-shot, no CoT, full article). Source: arxiv.org. Saturation forecast: Around February 2027. 14 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Mistral 7B Instruct (v0.3) | 80.8 | #6698 |
| 2 | Mistral Large 2 (Nov) Instruct (2411) | 68.5 | #1199 |
| 3 | GPT-5 Mini | 59.9 | #349 |
| 4 | Qwen 3 30B A3B 2507 Instruct | 58.5 | #1078 |
| 5 | Llama 3.3 70B Instruct [4bit] | 58 | #1221 |
| 6 | GPT-4.1 Nano | 55.9 | #1992 |
| 7 | Qwen 3 32B | 54.7 | #960 |
| 8 | Qwen 3 8B | 48.4 | #1838 |
| 9 | Phi-4 Mini Instruct | 42.6 | #6452 |
| 10 | GPT-4o Mini | 21.3 | #1627 |
| 11 | Llama 3.1 8B Instruct | 19 | #4277 |
| 12 | Llama 3.2 3B Instruct | 8.8 | #7185 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=epiqal-masked-input-reasoning · How It Works · Data refreshed daily, snapshot 2026-11-12.