ThermoQA - Component Analysis: leaderboard

ThermoQA tier 2: 101 multi-step analyses of single components such as turbines, compressors and heat exchangers, up to entropy generation and exergy destruction. Weighted step score in percent, higher is better.

Metric: Tier 2 weighted step score (%): 101 single-component problems (turbines, compressors, pumps, heat exchangers, boilers, mixing chambers, nozzles) at three analysis depths up to exergy destruction, steps weighted by engineering importance, over the 293 open-ended engineering thermodynamics problems of ThermoQA (ground truth computed with CoolProp 7.2.0 for water, R-134a and variable-cp air), zero-shot with a symbol = value unit answer format, each model at its stated reasoning setting, mean of three independent runs; values are extracted by regex and gpt-4.1-mini and a property counts as correct within 2 percent relative or 0.5 absolute tolerance (0.03 for quality, 0.02 for efficiencies and COP); higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 6 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Claude Opus 4.6Thinking92.1#114
2Gemini 3.1 Pro (Preview)90.8#102
3GPT-5.4High90.8#138
4ThermoQA DeepSeek-R1 (checkpoint unspecified)89.2
5Grok 4.20 Beta (0309)87.9#338
6MiniMax-M2.576.2#600

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=thermoqa-component-analysis · How It Works · Data refreshed daily, snapshot 2026-11-22.