LGMT - Consistent Accuracy: leaderboard
LGMT paper v1: 3,091 metamorphic groups from FOLIO, LogicNLI and ProverQA; each contains an original and first-order-logically equivalent follow-up (6,182 calls/model), zero-shot chain of thought. Twenty logic transformations. Invalid calls excluded and not counted as failures. Reasoning endpoints omit rejected sampling controls and escalate truncation budgets from 8,192 to 65,536 tokens. API generations absent from the paper remain source-qualified; no provider-default identity inference. Consistent Accuracy. Metric: Consistent Accuracy (%).
Metric: Consistent Accuracy (%). Source: arxiv.org. Saturation forecast: Around January 2027. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4.5 | 60.97 | #148 |
| 2 | GPT-5.2 | 56.84 | #115 |
| 3 | O4 Mini | 56.2 | #180 |
No result here: #3 Claude Opus 5.5, #5 GPT-6 Astra, #6 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=lgmt-consistent-accuracy · How It Works · Data refreshed daily, snapshot 2026-10-27.