TLA+-Bench (Configuration-Aware): leaderboard
TLA+-Bench (2026) asks models to write a TLA+ specification from a natural-language description and counts it correct only when the TLC model checker verifies it against the gold configuration, on 100 system specifications with one sample each. In this configuration-aware regime the prompt also supplies the constant and property names the configuration uses; score is the TLC-correct rate in percent, higher is better.
Metric: TLC-correct rate (%; configuration-aware regime, the prompt also supplies the constant and property names the configuration uses; a separate set of generations) on 100 model-checked gold TLA+ specifications (system category, trivial single-state fixtures removed), written from a GPT-5 declarative description, one sample each; correct when TLC model-checks the output against the reference configuration. Source: arxiv.org. Saturation forecast: Around October 2027. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.5 | 26 | #156 |
| 2 | Gemini 2.5 Pro | 19 | #266 |
| 3 | GPT-5 | 11 | #167 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=tla-plus-bench-configuration-aware · How It Works · Data refreshed daily, snapshot 2026-11-13.