ThermoQA - Cycle Analysis: leaderboard
ThermoQA tier 3: 82 full thermodynamic cycle analyses, from Rankine and Brayton variants to refrigeration and combined cycles. Weighted step score in percent, higher is better.
Metric: Tier 3 weighted step score (%): 82 full cycle analyses across ten cycle types (Rankine, Brayton, vapor-compression refrigeration, combined cycle), 15 to 60 weighted steps each, over the 293 open-ended engineering thermodynamics problems of ThermoQA (ground truth computed with CoolProp 7.2.0 for water, R-134a and variable-cp air), zero-shot with a symbol = value unit answer format, each model at its stated reasoning setting, mean of three independent runs; values are extracted by regex and gpt-4.1-mini and a property counts as correct within 2 percent relative or 0.5 absolute tolerance (0.03 for quality, 0.02 for efficiencies and COP); higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 6 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Claude Opus 4.6 | Thinking | 93.6 | #113 |
| 2 | GPT-5.4 | High | 89.7 | #135 |
| 3 | Gemini 3.1 Pro (Preview) | 87.5 | #99 | |
| 4 | ThermoQA DeepSeek-R1 (checkpoint unspecified) | 81 | ||
| 5 | Grok 4.20 Beta (0309) | 80.4 | #333 | |
| 6 | MiniMax-M2.5 | 52.7 | #600 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=thermoqa-cycle-analysis · How It Works · Data refreshed daily, snapshot 2026-11-21.