Paraphrase-Induced Output-Mode Collapse: leaderboard
When the substantive content of a request is rewritten, do large language models still answer in the format the original task asked for? We find that they often do not, even at temperature zero. On a 150-query evaluation over five compact 2025-era LLMs and four task types, we observe a systematic failure mode we call prompt-variant output-mode collapse: when a closed-form prompt asks for a bare la
Metric: Semantic Consistency Score (self-reported). Source: benchmarklist.com. Saturation forecast: Around December 2026. 5 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4.5 | 41.1 | #142 |
| 2 | Claude Haiku 4.5 | 34.4 | #280 |
| 3 | GPT-4o Mini | 28.5 | #599 |
| 4 | GPT-4.1 Mini | 26.2 | #355 |
| 5 | Gemini 2.5 Flash | 25.6 | #245 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=paraphrase-induced-output-mode-collapse · How It Works · Data refreshed daily, snapshot 2026-10-20.