NLU Corruption - Centered QNLI: leaderboard
This study tests natural-language inference with clean and corrupted inputs. Raw accuracy, answer-bias-centred accuracy and prompting conditions remain separate. The public ANLI-trained RoBERTa checkpoint is distinct from generic RoBERTa and the study’s locally fine-tuned encoders. Above-chance retention is not relabelled as accuracy. This board reports centered clean answer accuracy (%). Higher is better.
Metric: Centered clean answer accuracy (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 6 models tracked. Updated 14 Nov 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | OLMo 3 32B Base (Harness: NLU centered 5-shot, clean) | 91 | |
| 2 | OLMo 3 32B Base (Harness: NLU centered 5-shot, drop-NOUN) | 90.8 | |
| 3 | OLMo 3 32B Base (Harness: NLU centered 0-shot) | 88.5 | |
| 4 | OLMo 3 7B Base (Harness: NLU centered 5-shot, clean) | 87 | |
| 5 | OLMo 3 7B Base (Harness: NLU centered 5-shot, drop-NOUN) | 86.5 | |
| 6 | OLMo 3 7B Base (Harness: NLU centered 0-shot) | 82 |
Interactive version: theaggregate.ai/benchmark?slug=nlu-corruption-centered-qnli · How It Works · Data refreshed daily, snapshot 2026-11-16.